Abstract
Image completion is the problem of generating whole images from fragments only. It encompasses inpainting (generating a patch given its surrounding), reverse inpainting/extrapolation (generating the periphery given the central patch) as well as colorization (generating one or several channels given other ones). In this paper, we employ a deep network to perform image completion, with adversarial training as well as perceptual and completion losses, and call it the ``missing data encoder'' (MDE). We consider several configurations based on how the seed fragments are chosen. We show that training MDE for ``random extrapolation and colorization'' (MDE-REC), i.e. using random channel-independent fragments, allows a better capture of the image semantics and geometry. MDE training makes use of a novel ``hide-and-seek'' adversarial loss, where the discriminator seeks the original non-masked regions, while the generator tries to hide them. We validate our models both qualitatively and quantitatively on several datasets, showing their interest for image completion, unsupervised representation learning as well as face occlusion handling.
Abstract (translated)
图像完成是仅从片段生成整个图像的问题。它包括Inpainting（根据周围的情况生成一个补丁）、Reverse Inpainting/Extraction（根据中心补丁生成外围）和Colorize（根据其他通道生成一个或多个通道）。本文采用一个深度网络来完成图像的处理，并进行了对抗性训练和感知和完成损失，称之为“缺失数据编码器”（MDE）。我们根据如何选择种子片段来考虑几种配置。我们表明，为“随机外推和着色”（mde-rec）培训mde，即使用随机的独立于通道的片段，可以更好地捕获图像语义和几何图形。MDE训练利用了一种新的‘捉迷藏’的对抗性损失，其中鉴别器寻找原始的非掩蔽区域，而生成器试图隐藏它们。我们在多个数据集上定性和定量地验证了我们的模型，显示了它们对图像完成、无监督表示学习以及面部遮挡处理的兴趣。
URL
https://arxiv.org/abs/1905.01861
