Abstract
Free-form video inpainting is a very challenging task that could be widely used for video editing such as text removal. Existing patch-based methods could not handle non-repetitive structures such as faces, while directly applying image-based inpainting models to videos will result in temporal inconsistency (see <a href="http://bit.ly/2Fu1n6b">this http URL</a>). In this paper, we introduce a deep learn-ing based free-form video inpainting model, with proposed 3D gated convolutions to tackle the uncertainty of free-form masks and a novel Temporal PatchGAN loss to enhance temporal consistency. In addition, we collect videos and design a free-form mask generation algorithm to build the free-form video inpainting (FVI) dataset for training and evaluation of video inpainting models. We demonstrate the benefits of these components and experiments on both the FaceForensics and our FVI dataset suggest that our method is superior to existing ones.
Abstract (translated)
自由格式视频绘制是一项非常具有挑战性的任务，可以广泛应用于视频编辑，如文本删除。现有的基于修补程序的方法无法处理非重复结构（如面），而直接将基于图像的inpainting模型应用于视频将导致时间不一致（请参见<a href=“http://bit.ly/2fu1n6b”>this http url</a>）。本文介绍了一种基于深度学习的自由格式视频修复模型，提出了一种基于三维门控卷积的方法来解决自由格式掩模的不确定性，并提出了一种新的增强时间一致性的时间补丁丢失方法。此外，我们还收集视频并设计了一个自由格式的遮罩生成算法，以构建自由格式的视频输入（FVI）数据集，用于视频输入模型的培训和评估。我们证明了这些组件的好处，并对人脸取证和我们的fvi数据集进行了实验，表明我们的方法优于现有的方法。
URL
https://arxiv.org/abs/1904.10247
