Abstract
This paper proposes a mask optimization method for improving the quality of object removal using image inpainting. While many inpainting methods are trained with a set of random masks, a target for inpainting may be an object, such as a person, in many realistic scenarios. This domain gap between masks in training and inference images increases the difficulty of the inpainting task. In our method, this domain gap is resolved by training the inpainting network with object masks extracted by segmentation, and such object masks are also used in the inference step. Furthermore, to optimize the object masks for inpainting, the segmentation network is connected to the inpainting network and end-to-end trained to improve the inpainting performance. The effect of this end-to-end training is further enhanced by our mask expansion loss for achieving the trade-off between large and small masks. Experimental results demonstrate the effectiveness of our method for better object removal using image inpainting.
Abstract (translated)
本文提出了一种使用图像修复方法来提高物体移除质量的口罩优化方法。虽然许多修复方法使用一组随机掩码进行训练，但在许多现实场景中，修复的目标可能是物体，例如人。训练图和推理图之间域差的存在增加了修复任务的难度。在我们的方法中，通过通过分割提取物体掩码来训练修复网络，使得修复网络使用的物体掩码与推理步骤使用的物体掩码相同。此外，为了优化用于修复的物体掩码，分割网络与修复网络相连，端到端训练以提高修复性能。通过扩展掩码损失实现大和小掩码之间的权衡，进一步增强了端到端训练的效果。实验结果证明了我们的修复方法在图像修复中的有效性。
URL
https://arxiv.org/abs/2403.15849
