Abstract
The efficiency of video computer vision system remains a challenging task due to the high temporal redundancy inside a video. Existing works have been proposed for efficient vision computer vision. However, they do not fully reduce the temporal redundancy and neglect the front end computation overhead. In this paper, we propose an efficient video computer vision system. First, image signal processor is removed and Bayer-format data is directly fed into video computer vision models, thus saving the front end computation. Second, instead of optical flow models and video codecs, a fast block matching-based motion estimation algorithm is proposed specifically for efficient video computer vision, with a MV refinement module. To correct the error, context-aware block refinement network is introduced to refine regions with large error. To further balance the accuracy and efficiency, a frame selection strategy is employed. Experiments on multiple video computer vision tasks demonstrate that our method achieves significant acceleration with slight performance loss.
Abstract (translated)
视频计算机视觉系统的效率仍然是一个挑战，因为视频内部存在较高的时间冗余。尽管已有研究提出了一些高效的方法来解决这一问题，但它们并未完全消除时间冗余，并且忽视了前端计算的开销。在本文中，我们提出了一种高效的视频计算机视觉系统。首先，移除图像信号处理器，并直接将Bayer格式数据输入到视频计算机视觉模型中，从而节省前端计算资源。其次，我们提出一种基于快速块匹配的运动估计算法以取代现有的光学流模型和视频编解码器，同时引入了MV（运动向量）精炼模块。为了纠正错误，我们还引入了一种上下文感知的块精炼网络来修正误差较大的区域。为平衡准确性和效率，我们采用了帧选择策略。在多个视频计算机视觉任务上的实验表明，我们的方法能够在性能略有下降的情况下实现显著加速。
URL
https://arxiv.org/abs/2508.05990
