Abstract
Image restoration aims to recover details and enhance contrast in degraded images. With the growing demand for high-quality imaging (\textit{e.g.}, 4K and 8K), achieving a balance between restoration quality and computational efficiency has become increasingly critical. Existing methods, primarily based on CNNs, Transformers, or their hybrid approaches, apply uniform deep representation extraction across the image. However, these methods often struggle to effectively model long-range dependencies and largely overlook the spatial characteristics of image degradation (regions with richer textures tend to suffer more severe damage), making it hard to achieve the best trade-off between restoration quality and efficiency. To address these issues, we propose a novel texture-aware image restoration method, TAMambaIR, which simultaneously perceives image textures and achieves a trade-off between performance and efficiency. Specifically, we introduce a novel Texture-Aware State Space Model, which enhances texture awareness and improves efficiency by modulating the transition matrix of the state-space equation and focusing on regions with complex textures. Additionally, we design a {Multi-Directional Perception Block} to improve multi-directional receptive fields while maintaining low computational overhead. Extensive experiments on benchmarks for image super-resolution, deraining, and low-light image enhancement demonstrate that TAMambaIR achieves state-of-the-art performance with significantly improved efficiency, establishing it as a robust and efficient framework for image restoration.
Abstract (translated)
图像恢复的目标是还原受损图像中的细节并增强对比度。随着对高质量成像（例如4K和8K）需求的增长，实现修复质量和计算效率之间的平衡变得越来越关键。现有方法主要基于卷积神经网络(CNNs)、变换器(Transformer)，或两者的混合方法，在整个图像上应用统一的深度表示提取技术。然而，这些方法往往难以有效地建模长程依赖关系，并且严重忽视了图像降质的空间特性（纹理更丰富的区域通常遭受更严重的损坏），从而很难在修复质量和效率之间取得最佳平衡。 为了应对这些问题，我们提出了一种新型的基于纹理感知的图像恢复方法——TAMambaIR。该方法能够同时感知图像中的纹理特征并在性能与计算效率之间实现权衡。具体来说，我们引入了一个新颖的“基于纹理的状态空间模型”（Texture-Aware State Space Model），通过调节状态空间方程的转换矩阵来增强对纹理的认知，并专注于复杂纹理区域以提高效率。此外，我们设计了一种“多向感知模块”，旨在改善图像的多方向感受野，同时保持较低的计算开销。 在超分辨率、去雨和低光照图像增强等基准测试上的广泛实验表明，TAMambaIR能够实现最先进的性能，并且显著提高了效率，从而确立了其作为稳健而高效的图像恢复框架的地位。
URL
https://arxiv.org/abs/2501.16583
