Abstract
Large pre-trained video diffusion models excel in video frame interpolation but struggle to generate high fidelity frames due to reliance on intrinsic generative priors, limiting detail preservation from start and end frames. Existing methods often depend on motion control for temporal consistency, yet dense optical flow is error-prone, and sparse points lack structural context. In this paper, we propose FC-VFI for faithful and consistent video frame interpolation, supporting \(4\times\)x and \(8\times\) interpolation, boosting frame rates from 30 FPS to 120 and 240 FPS at \(2560\times 1440\)resolution while preserving visual fidelity and motion consistency. We introduce a temporal modeling strategy on the latent sequences to inherit fidelity cues from start and end frames and leverage semantic matching lines for structure-aware motion guidance, improving motion consistency. Furthermore, we propose a temporal difference loss to mitigate temporal inconsistencies. Extensive experiments show FC-VFI achieves high performance and structural integrity across diverse scenarios.
Abstract (translated)
大型预训练视频扩散模型在视频帧插值方面表现出色，但在生成高保真度的帧时却由于依赖于内在的生成先验而难以保留细节。现有的方法通常依靠运动控制来保持时间一致性，然而密集光流容易出错，稀疏点则缺乏结构上下文信息。在这篇论文中，我们提出了FC-VFI（Faithful and Consistent Video Frame Interpolation）模型，旨在实现忠实且一致的视频帧插值，并支持4倍和8倍的插值率，在2560×1440分辨率下将帧率从30 FPS提升至120 FPS及240 FPS的同时保持视觉保真度与运动一致性。我们引入了一种时序建模策略，用于在潜在序列中继承起始帧和结束帧的保真度线索，并利用语义匹配线来为结构感知运动引导提供支持，从而提高运动一致性。此外，我们还提出了一种时间差异损失函数以减少时间上的不一致现象。通过广泛的实验验证，FC-VFI模型在各种场景下均展现出高水准的表现及结构完整性。
URL
https://arxiv.org/abs/2603.04899
