Abstract
Large pretrained diffusion models have significantly enhanced the quality of generated videos, and yet their use in real-time streaming remains limited. Autoregressive models offer a natural framework for sequential frame synthesis but require heavy computation to achieve high fidelity. Diffusion distillation can compress these models into efficient few-step variants, but existing video distillation approaches largely adapt image-specific methods that neglect temporal dependencies. These techniques often excel in image generation but underperform in video synthesis, exhibiting reduced motion coherence, error accumulation over long sequences, and a latency-quality trade-off. We identify two factors that result in these limitations: insufficient utilization of temporal context during step reduction and implicit prediction of subsequent noise levels in next-chunk prediction (i.e., exposure bias). To address these issues, we propose Diagonal Distillation, which operates orthogonally to existing approaches and better exploits temporal information across both video chunks and denoising steps. Central to our approach is an asymmetric generation strategy: more steps early, fewer steps later. This design allows later chunks to inherit rich appearance information from thoroughly processed early chunks, while using partially denoised chunks as conditional inputs for subsequent synthesis. By aligning the implicit prediction of subsequent noise levels during chunk generation with the actual inference conditions, our approach mitigates error propagation and reduces oversaturation in long-range sequences. We further incorporate implicit optical flow modeling to preserve motion quality under strict step constraints. Our method generates a 5-second video in 2.61 seconds (up to 31 FPS), achieving a 277.3x speedup over the undistilled model.
Abstract (translated)
大型预训练扩散模型在生成视频的质量上有了显著的提升，然而它们在实时流媒体中的应用仍然受到限制。自回归模型为序列帧合成提供了一个自然框架，但要达到高保真度则需要大量计算资源。扩散蒸馏可以将这些大型模型压缩成高效的几步变体版本，但是现有的视频蒸馏方法大多借鉴了特定于图像的方法，忽视了时间依赖性。虽然这些技术在图像生成中表现出色，但在视频合成中的表现较差，表现为运动连贯性降低、长时间序列中的误差积累以及延迟与质量之间的权衡问题。 我们发现导致这些问题的两个关键因素是：减少步骤过程中对时间上下文利用不足和下一片段预测（即曝光偏差）中隐式预测随后噪声水平的问题。为了解决这些问题，我们提出了对角蒸馏方法，这种方法与现有的方法形成了补充，并能更好地利用视频片段以及去噪步之间的时空信息。我们的方法的核心是对称生成策略：早期步骤多、后期步骤少。这种设计使后续片段能够从先前充分处理的片段中继承丰富的视觉信息，同时将部分去噪后的片段作为条件输入用于后续合成。通过在片段生成期间与实际推断条件对齐隐式预测随后噪声水平，我们的方法减轻了误差传播，并减少了长时间序列中的过度饱和问题。 此外，我们还结合了隐式光流建模以在严格的步骤限制下保持运动质量。我们的方法可以在2.61秒内生成5秒钟的视频（最高可达31帧每秒），比未蒸馏模型的速度提高了277.3倍。
URL
https://arxiv.org/abs/2603.09488
