Abstract
In bandwidth-limited online video streaming, videos are usually downsampled and compressed. Although recent online video super-resolution (online VSR) approaches achieve promising results, they are still compute-intensive and fall short of real-time processing at higher resolutions, due to complex motion estimation for alignment and redundant processing of consecutive frames. To address these issues, we propose a compressed-domain-aware network (CDA-VSR) for online VSR, which utilizes compressed-domain information, including motion vectors, residual maps, and frame types to balance quality and efficiency. Specifically, we propose a motion-vector-guided deformable alignment module that uses motion vectors for coarse warping and learns only local residual offsets for fine-tuned adjustments, thereby maintaining accuracy while reducing computation. Then, we utilize a residual map gated fusion module to derive spatial weights from residual maps, suppressing mismatched regions and emphasizing reliable details. Further, we design a frame-type-aware reconstruction module for adaptive compute allocation across frame types, balancing accuracy and efficiency. On the REDS4 dataset, our CDA-VSR surpasses the state-of-the-art method TMP, with a maximum PSNR improvement of 0.13 dB while delivering more than double the inference speed. The code will be released at this https URL.
Abstract (translated)
在带宽受限的在线视频流媒体中，通常会将视频进行降采样和压缩。尽管最近的一些在线视频超分辨率（online VSR）方法取得了令人满意的结果，但由于复杂的运动估计以实现对齐以及连续帧处理中的冗余计算，这些方法仍然需要大量的计算资源，并且在高分辨率下的实时处理方面仍存在不足。 为了解决这些问题，我们提出了一种压缩域感知网络（CDA-VSR），该网络利用包括运动向量、残差图和帧类型在内的压缩域信息，在保证质量的同时提高效率。具体来说，我们提出了一个基于运动矢量引导的可变形对齐模块，该模块使用运动向量进行粗略扭曲，并仅学习局部残差偏移以实现精细调整，从而在减少计算需求的情况下保持准确性。此外，我们设计了一个从残差图中推导空间权重的残差图门控融合模块，抑制不匹配区域并强调可靠细节。另外，为了在不同类型的帧之间进行自适应计算分配，平衡准确性和效率，我们还设计了一个基于帧类型感知重建模块。 在RED-S4数据集上，我们的CDA-VSR方法超越了最新的TMP方法，在最大PSNR改善达到0.13分贝的同时，推理速度提高了两倍以上。代码将在以下链接发布：[此处插入URL]。
URL
https://arxiv.org/abs/2603.07694
