Abstract
Optical flow models trained on high-quality data often degrade severely when confronted with real-world corruptions such as blur, noise, and compression artifacts. To overcome this limitation, we formulate Degradation-Aware Optical Flow, a new task targeting accurate dense correspondence estimation from real-world corrupted videos. Our key insight is that the intermediate representations of image restoration diffusion models are inherently corruption-aware but lack temporal awareness. To address this limitation, we lift the model to attend across adjacent frames via full spatio-temporal attention, and empirically demonstrate that the resulting features exhibit zero-shot correspondence capabilities. Based on this finding, we present DA-Flow, a hybrid architecture that fuses these diffusion features with convolutional features within an iterative refinement framework. DA-Flow substantially outperforms existing optical flow methods under severe degradation across multiple benchmarks.
Abstract (translated)
在高质量数据上训练的光流模型，在面对模糊、噪声和压缩伪影等真实世界退化时，性能往往会严重下降。为克服这一限制，我们提出了“退化感知光流”（Degradation-Aware Optical Flow）这一新任务，旨在从真实世界退化的视频中实现准确的稠密对应估计。我们的核心见解是：图像修复扩散模型的中间表示天然具备退化感知能力，但缺乏时序感知能力。为解决此问题，我们通过全时空注意力机制使模型能够跨相邻帧进行关注，并实证表明由此产生的特征具有零样本对应能力。基于这一发现，我们提出了DA-Flow——一种混合架构，它在迭代精炼框架内融合了这些扩散特征与卷积特征。在多个基准测试的严重退化场景下，DA-Flow的表现均显著优于现有光流方法。
URL
https://arxiv.org/abs/2603.23499
