Abstract
The scarcity of ground-truth labels poses one major challenge in developing optical flow estimation models that are both generalizable and robust. While current methods rely on data augmentation, they have yet to fully exploit the rich information available in labeled video sequences. We propose OCAI, a method that supports robust frame interpolation by generating intermediate video frames alongside optical flows in between. Utilizing a forward warping approach, OCAI employs occlusion awareness to resolve ambiguities in pixel values and fills in missing values by leveraging the forward-backward consistency of optical flows. Additionally, we introduce a teacher-student style semi-supervised learning method on top of the interpolated frames. Using a pair of unlabeled frames and the teacher model's predicted optical flow, we generate interpolated frames and flows to train a student model. The teacher's weights are maintained using Exponential Moving Averaging of the student. Our evaluations demonstrate perceptually superior interpolation quality and enhanced optical flow accuracy on established benchmarks such as Sintel and KITTI.
Abstract (translated)
真实标签的稀缺性在开发既具有普适性又具有韧性的光流估计模型方面构成了一个主要挑战。虽然当前方法依赖于数据增强，但它们尚未充分利用已有的带标签视频序列中存在的丰富信息。我们提出了一种名为OCAI的方法，通过在光流之间生成中间视频帧来支持鲁棒帧插值。利用前膨胀方法，OCAI通过利用光流的前后一致性来解决像素值的不确定性，并利用光流的信息来填充缺失值。此外，我们在插值帧上引入了一种师生风格的中半监督学习方法。通过使用带标签帧和学生模型的预测光流，我们生成插值帧和光流以训练学生模型。学生模型的权重通过学生模型的学生指数进行维持。我们在Sintel和KITTI等已建立的标准基准上进行评估，证明了我们的插值质量和光流准确性的提高。
URL
https://arxiv.org/abs/2403.18092
