Abstract
Video matting is crucial for applications such as film production and virtual reality, yet deploying its computationally intensive models on resource-constrained devices presents challenges. Quantization is a key technique for model compression and acceleration. As an efficient approach, Post-Training Quantization (PTQ) is still in its nascent stages for video matting, facing significant hurdles in maintaining accuracy and temporal coherence. To address these challenges, this paper proposes a novel and general PTQ framework specifically designed for video matting models, marking, to the best of our knowledge, the first systematic attempt in this domain. Our contributions include: (1) A two-stage PTQ strategy that combines block-reconstruction-based optimization for fast, stable initial quantization and local dependency capture, followed by a global calibration of quantization parameters to minimize accuracy loss. (2) A Statistically-Driven Global Affine Calibration (GAC) method that enables the network to compensate for cumulative statistical distortions arising from factors such as neglected BN layer effects, even reducing the error of existing PTQ methods on video matting tasks up to 20%. (3) An Optical Flow Assistance (OFA) component that leverages temporal and semantic priors from frames to guide the PTQ process, enhancing the model's ability to distinguish moving foregrounds in complex scenes and ultimately achieving near full-precision performance even under ultra-low-bit quantization. Comprehensive quantitative and visual results show that our PTQ4VM achieves the state-of-the-art accuracy performance across different bit-widths compared to the existing quantization methods. We highlight that the 4-bit PTQ4VM even achieves performance close to the full-precision counterpart while enjoying 8x FLOP savings.
Abstract (translated)
视频抠图（Video Matting）在电影制作和虚拟现实等领域中至关重要，但要在计算资源有限的设备上部署其计算密集型模型存在挑战。量化是一种关键的技术，用于模型压缩和加速。作为一种高效的途径，后训练量化（Post-Training Quantization, PTQ）在视频抠图领域尚处于起步阶段，并且面临着保持精度和时间一致性的重要障碍。为了解决这些问题，本文提出了一种专为视频抠图模型设计的新颖而通用的PTQ框架，据我们所知，这是该领域的首次系统性尝试。我们的贡献包括： 1. 一种两阶段PTQ策略，结合基于块重构的优化方法进行快速、稳定的初步量化和局部依赖项捕获，随后通过全局校准量化参数来最小化精度损失。 2. 统计驱动全局仿射校准（GAC）方法，使网络能够补偿由于忽略BN层效应等因素产生的累积统计失真，甚至将现有PTQ方法在视频抠图任务中的误差减少高达20%。 3. 光流辅助（OFA）组件，利用帧间的时间和语义先验引导量化过程，增强模型区分复杂场景中移动前景的能力，并最终实现即使在超低位宽量化下也能达到接近全精度的表现。 综合定量和视觉结果表明，我们的PTQ4VM方法在不同比特宽度下的准确性表现优于现有的量化方法。特别值得注意的是，4位的PTQ4VM甚至实现了与全精度模型相当的性能，同时节省了8倍的计算量（FLOP）。
URL
https://arxiv.org/abs/2506.10840
