Abstract
Dynamic image degradations, including noise, blur and lighting inconsistencies, pose significant challenges in image restoration, often due to sensor limitations or adverse environmental conditions. Existing Deep Unfolding Networks (DUNs) offer stable restoration performance but require manual selection of degradation matrices for each degradation type, limiting their adaptability across diverse scenarios. To address this issue, we propose the Vision-Language-guided Unfolding Network (VLU-Net), a unified DUN framework for handling multiple degradation types simultaneously. VLU-Net leverages a Vision-Language Model (VLM) refined on degraded image-text pairs to align image features with degradation descriptions, selecting the appropriate transform for target degradation. By integrating an automatic VLM-based gradient estimation strategy into the Proximal Gradient Descent (PGD) algorithm, VLU-Net effectively tackles complex multi-degradation restoration tasks while maintaining interpretability. Furthermore, we design a hierarchical feature unfolding structure to enhance VLU-Net framework, efficiently synthesizing degradation patterns across various levels. VLU-Net is the first all-in-one DUN framework and outperforms current leading one-by-one and all-in-one end-to-end methods by 3.74 dB on the SOTS dehazing dataset and 1.70 dB on the Rain100L deraining dataset.
Abstract (translated)
动态图像退化，包括噪声、模糊和光照不一致等问题，在图像恢复领域提出了重大挑战，这些问题通常由传感器限制或不利的环境条件引起。现有的深度解卷网络（DUNs）提供了稳定的恢复性能，但需要为每种降质类型手动选择降质矩阵，这限制了它们在各种场景中的适应性。为了解决这个问题，我们提出了一种基于视觉-语言引导的解卷网络（VLU-Net），这是一种统一的DUN框架，能够同时处理多种退化类型。 VLU-Net利用一个经过退化图像-文本对训练的视觉-语言模型（VLM）来将图像特征与降质描述进行对齐，并选择适当的转换以适应目标降质。通过在近似梯度下降算法中集成一种基于VLM的自动梯度估计策略，VLU-Net能够有效处理复杂的多退化恢复任务，同时保持可解释性。 此外，我们设计了一种分层特征解卷结构，增强了VLU-Net框架，在不同层次上高效地合成退化模式。VLU-Net是首个全功能的DUN框架，并且在SOTS去雾数据集和Rain100L除雨数据集中分别优于当前领先的逐一处理方法以及端到端一体化方法3.74 dB和1.70 dB。
URL
https://arxiv.org/abs/2503.16930
