Abstract
Video inpainting, which aims at filling in missing regions of a video, remains challenging due to the difficulty of preserving the precise spatial and temporal coherence of video contents. In this work we propose a novel flow-guided video inpainting approach. Rather than filling in the RGB pixels of each frame directly, we consider video inpainting as a pixel propagation problem. We first synthesize a spatially and temporally coherent optical flow field across video frames using a newly designed Deep Flow Completion network. Then the synthesized flow field is used to guide the propagation of pixels to fill up the missing regions in the video. Specifically, the Deep Flow Completion network follows a coarse-to-fine refinement to complete the flow fields, while their quality is further improved by hard flow example mining. Following the guide of the completed flow, the missing video regions can be filled up precisely. Our method is evaluated on DAVIS and YouTube-VOS datasets qualitatively and quantitatively, achieving the state-of-the-art performance in terms of inpainting quality and speed.
Abstract (translated)
由于难以保持视频内容的精确空间和时间一致性，旨在填补视频缺失区域的视频修复仍然具有挑战性。在这项工作中，我们提出了一种新的流引导视频修复方法。我们不直接填充每帧的RGB像素，而是将视频输入视为像素传播问题。我们首先使用一个新设计的深流完成网络合成一个跨越视频帧的时空相干光流场。然后利用合成流场引导像素的传播，填充视频中缺失的区域。具体地说，深流完井网络采用粗到细的细化来完成流场，而硬流实例挖掘则进一步提高了其质量。在完成流程的引导下，可以精确填充缺失的视频区域。我们的方法在Davis和YouTube VOS数据集上进行了定性和定量的评估，在喷漆质量和速度方面达到了最先进的性能。
URL
https://arxiv.org/abs/1905.02884
