Abstract
Benefiting from powerful convolutional neural networks (CNNs), learning-based image inpainting methods have made significant breakthroughs over the years. However, some nature of CNNs (e.g. local prior, spatially shared parameters) limit the performance in the face of broken images with diverse and complex forms. Recently, a class of attention-based network architectures, called transformer, has shown significant performance on natural language processing fields and high-level vision tasks. Compared with CNNs, attention operators are better at long-range modeling and have dynamic weights, but their computational complexity is quadratic in spatial resolution, and thus less suitable for applications involving higher resolution images, such as image inpainting. In this paper, we design a novel attention linearly related to the resolution according to Taylor expansion. And based on this attention, a network called $T$-former is designed for image inpainting. Experiments on several benchmark datasets demonstrate that our proposed method achieves state-of-the-art accuracy while maintaining a relatively low number of parameters and computational complexity. The code can be found at \href{this https URL}{this http URL\_image\_inpainting}
Abstract (translated)
得益于强大的卷积神经网络(CNNs)，基于学习的图像处理方法在过去取得了重大突破。然而，CNNs的一些特性(例如局部先验、空间共享参数)在面对各种复杂和多样化的破碎图像时限制了性能。最近，一种名为Transformer的注意力型网络架构类表现出了在自然语言处理领域和高层次视觉任务中的重大表现。与CNNs相比，注意力操作在长距离建模方面表现更好，并具有动态权重，但它们的计算复杂度在空间分辨率上是平方的，因此不太适用于包括高分辨率图像的应用程序，如图像修复。在本文中，我们根据泰勒展开设计了一份与分辨率线性相关的新的注意力。基于这个注意力，我们设计了名为$T$-former的图像修复网络。对多个基准数据集进行的实验证明，我们提出的方法在保持相对低参数和计算复杂度的情况下实现了最先进的精度。代码可在\href{this https URL}{this http URL\_image\_inpainting}中找到。
URL
https://arxiv.org/abs/2305.07239
