Abstract
Image inpainting has earned substantial progress, owing to the encoder-and-decoder pipeline, which is benefited from the Convolutional Neural Networks (CNNs) with convolutional downsampling to inpaint the masked regions semantically from the known regions within the encoder, coupled with an upsampling process from the decoder for final inpainting output. Recent studies intuitively identify the high-frequency structure and low-frequency texture to be extracted by CNNs from the encoder, and subsequently for a desirable upsampling recovery. However, the existing arts inevitably overlook the information loss for both structure and texture feature maps during the convolutional downsampling process, hence suffer from a non-ideal upsampling output. In this paper, we systematically answer whether and how the structure and texture feature map can mutually help to alleviate the information loss during the convolutional downsampling. Given the structure and texture feature maps, we adopt the statistical normalization and denormalization strategy for the reconstruction guidance during the convolutional downsampling process. The extensive experimental results validate its advantages to the state-of-the-arts over the images from low-to-high resolutions including 256*256 and 512*512, especially holds by substituting all the encoders by ours. Our code is available at this https URL
Abstract (translated)
图像修复技术取得了显著进步，这主要归功于编码器-解码器管道的使用，该管道利用了具有卷积下采样功能的卷积神经网络（CNN），以从编码器中已知区域语义化地修复被遮罩的区域，并通过解码器中的上采样过程生成最终的图像修复输出。最近的研究直观地发现，CNN在编码过程中能够分别提取高频结构和低频纹理信息，并进一步进行所需的上采样恢复。然而，现有的技术不可避免地忽视了卷积下采样过程中对于结构和纹理特征图的信息丢失问题，从而导致非理想的上采样结果。在这篇文章中，我们系统性地探讨了如何通过结构和纹理特征图相互帮助来减轻在卷积下采样过程中的信息损失。基于这种策略，我们在给定的结构和纹理特征图基础上采用了统计归一化和反归一化的重建引导方法来进行卷积下采样的处理。广泛的实验结果验证了这种方法相对于现有的图像修复技术（包括从低分辨率到高分辨率如256*256和512*512的各种图像）具有优势，特别是在全部使用我们提出的编码器时尤为明显。我们的代码可以在这个链接上找到：[https URL]
URL
https://arxiv.org/abs/2602.03013
