Abstract
While many diffusion models have achieved impressive results in real-world video super-resolution (Real-VSR) by generating rich and realistic details, their reliance on multi-step sampling leads to slow inference. One-step networks like SeedVR2, DOVE, and DLoRAL alleviate this through condensing generation into one single step, yet they remain heavy, with billions of parameters and multi-second latency. Recent adversarial diffusion compression (ADC) offers a promising path via pruning and distilling these models into a compact AdcSR network, but directly applying it to Real-VSR fails to balance spatial details and temporal consistency due to its lack of temporal awareness and the limitations of standard adversarial learning. To address these challenges, we propose an improved ADC method for Real-VSR. Our approach distills a large diffusion Transformer (DiT) teacher DOVE equipped with 3D spatio-temporal attentions, into a pruned 2D Stable Diffusion (SD)-based AdcSR backbone, augmented with lightweight 1D temporal convolutions, achieving significantly higher efficiency. In addition, we introduce a dual-head adversarial distillation scheme, in which discriminators in both pixel and feature domains explicitly disentangle the discrimination of details and consistency into two heads, enabling both objectives to be effectively optimized without sacrificing one for the other. Experiments demonstrate that the resulting compressed AdcVSR model reduces complexity by 95% in parameters and achieves an 8$\times$ acceleration over its DiT teacher DOVE, while maintaining competitive video quality and efficiency.
Abstract (translated)
虽然许多扩散模型在现实世界的视频超分辨率（Real-VSR）任务中通过生成丰富和逼真的细节取得了令人印象深刻的结果，但它们依赖于多步骤采样导致推理速度慢。类似于SeedVR2、DOVE和DLoRAL的一步网络通过将生成过程压缩为单一步骤来缓解这一问题，但仍保持庞大体量，拥有数十亿参数且存在数秒延迟的问题。最近的对抗性扩散压缩（ADC）方法通过修剪并蒸馏这些模型至紧凑型AdcSR网络提供了一种有希望的道路，但直接应用于Real-VSR时由于缺乏时间感知能力及标准对抗学习的限制而无法平衡空间细节与时间一致性。 为了解决这些问题，我们提出了一种改进的ADC方法用于Real-VSR。我们的方法从一个大型扩散Transformer（DiT）教师网络DOVE开始，该网络配备了3D时空注意力机制，并将它蒸馏至一个轻量级修剪后的2D稳定扩散（SD）基础AdcSR模型上，同时增加了轻量的一维时间卷积，从而显著提高了效率。此外，我们引入了一个双头对抗性蒸馏方案，在像素域和特征域中明确区分细节与一致性的鉴别器被拆分为两个头部，使得两者都能在不牺牲其中一个目标的情况下有效优化。 实验表明，压缩后的AdcVSR模型相较于其DiT教师DOVE在参数复杂度上减少了95%，实现了8倍的加速，同时保持了竞争性的视频质量和效率。
URL
https://arxiv.org/abs/2603.00458
