Abstract
Video Diffusion Models (VDMs) can generate high-quality videos, but often struggle with producing temporally coherent motion. Optical flow supervision is a promising approach to address this, with prior works commonly employing warping-based strategies that avoid explicit flow matching. In this work, we explore an alternative formulation, FlowLoss, which directly compares flow fields extracted from generated and ground-truth videos. To account for the unreliability of flow estimation under high-noise conditions in diffusion, we propose a noise-aware weighting scheme that modulates the flow loss across denoising steps. Experiments on robotic video datasets suggest that FlowLoss improves motion stability and accelerates convergence in early training stages. Our findings offer practical insights for incorporating motion-based supervision into noise-conditioned generative models.
Abstract (translated)
视频扩散模型（VDMs）能够生成高质量的视频，但在产生连贯的时间运动方面常常遇到困难。光学流监督是一种有前景的方法来解决这一问题，之前的工作通常采用基于变形的策略，避免了显式的流匹配。在这项工作中，我们探索了一种替代方案FlowLoss，它直接比较从生成和真实视频中提取的流动场。为了应对扩散过程中的高噪声条件下流动估计不可靠的问题，我们提出了一种噪声感知加权方案，该方案调节去噪步骤中的流动损失。在机器人视频数据集上的实验表明，FlowLoss提高了运动稳定性，并且在早期训练阶段加速了收敛速度。我们的研究结果为将基于运动的监督纳入噪声条件下的生成模型提供了实用见解。
URL
https://arxiv.org/abs/2504.14535
