Abstract
In the image acquisition process, various forms of degradation, including noise, haze, and rain, are frequently introduced. These degradations typically arise from the inherent limitations of cameras or unfavorable ambient conditions. To recover clean images from degraded versions, numerous specialized restoration methods have been developed, each targeting a specific type of degradation. Recently, all-in-one algorithms have garnered significant attention by addressing different types of degradations within a single model without requiring prior information of the input degradation type. However, these methods purely operate in the spatial domain and do not delve into the distinct frequency variations inherent to different degradation types. To address this gap, we propose an adaptive all-in-one image restoration network based on frequency mining and modulation. Our approach is motivated by the observation that different degradation types impact the image content on different frequency subbands, thereby requiring different treatments for each restoration task. Specifically, we first mine low- and high-frequency information from the input features, guided by the adaptively decoupled spectra of the degraded image. The extracted features are then modulated by a bidirectional operator to facilitate interactions between different frequency components. Finally, the modulated features are merged into the original input for a progressively guided restoration. With this approach, the model achieves adaptive reconstruction by accentuating the informative frequency subbands according to different input degradations. Extensive experiments demonstrate that the proposed method achieves state-of-the-art performance on different image restoration tasks, including denoising, dehazing, deraining, motion deblurring, and low-light image enhancement. Our code is available at this https URL.
Abstract (translated)
在图像采集过程中，经常引入各种形式的衰减，包括噪声、雾和雨。这些衰减通常来源于相机本身或不利环境条件的固有局限性。为了从衰减版本中恢复干净的图像，已经开发了许多专门的修复方法，每个方法都针对特定的衰减类型。最近，全息算法在解决单个模型内不同类型的衰减方面得到了显著关注，而不需要先前的输入衰减类型信息。然而，这些方法仅在空间域操作，而没有深入研究不同衰减类型固有的不同频率变化。为了填补这一空白，我们提出了一个基于频率挖掘和调制的自适应全息图像修复网络。我们的方法源于观察到不同衰减类型对不同频率子带图像内容的影响，因此为每个修复任务需要不同的处理方法。具体来说，我们首先从输入特征中挖掘低频和高频信息，并基于衰减图像的适应解耦光谱进行指导。提取的特征随后被双向操作模态进行模调，以促进不同频率分量之间的相互作用。最后，模调后的特征合并到原始输入以实现渐进引导修复。通过这种方法，模型通过根据不同输入衰减类型的信息增强有用的频率子带，实现自适应重建。大量实验证明，与现有方法相比，所提出的修复方法在各种图像修复任务上实现了最先进的性能，包括去噪、消雾、去雨、运动模糊和低光图像增强。我们的代码可在此处访问：https://www.xxx.com。
URL
https://arxiv.org/abs/2403.14614
