Abstract
With the development of deep learning, numerous methods for low-light image enhancement (LLIE) have demonstrated remarkable performance. Mainstream LLIE methods typically learn an end-to-end mapping based on pairs of low-light and normal-light images. However, normal-light images under varying illumination conditions serve as reference images, making it difficult to define a ``perfect'' reference image This leads to the challenge of reconciling metric-oriented and visual-friendly results. Recently, many cross-modal studies have found that side information from other related modalities can guide visual representation learning. Based on this, we introduce a Natural Language Supervision (NLS) strategy, which learns feature maps from text corresponding to images, offering a general and flexible interface for describing an image under different illumination. However, image distributions conditioned on textual descriptions are highly multimodal, which makes training difficult. To address this issue, we design a Textual Guidance Conditioning Mechanism (TCM) that incorporates the connections between image regions and sentence words, enhancing the ability to capture fine-grained cross-modal cues for images and text. This strategy not only utilizes a wider range of supervised sources, but also provides a new paradigm for LLIE based on visual and textual feature alignment. In order to effectively identify and merge features from various levels of image and textual information, we design an Information Fusion Attention (IFA) module to enhance different regions at different levels. We integrate the proposed TCM and IFA into a Natural Language Supervision network for LLIE, named NaLSuper. Finally, extensive experiments demonstrate the robustness and superior effectiveness of our proposed NaLSuper.
Abstract (translated)
随着深度学习的发展，许多低光图像增强（LLIE）方法展现了卓越的性能。主流的LLIE方法通常基于低光照和正常光照条件下的成对图像来学习端到端映射。然而，由于不同照明条件下正常的光照图像用作参考图像，定义一个“完美”的参考图像变得困难。这导致了指标导向与视觉友好的结果之间的矛盾挑战。最近，许多跨模态研究表明，来自其他相关模态的侧面信息可以引导视觉表示学习。基于此，我们引入了一种自然语言监督（NLS）策略，该策略从对应于图像的文本中学习特征图，为不同光照条件下描述图像提供了一个通用且灵活的接口。然而，根据文本描述条件下的图像分布是多模态的，这使得训练变得困难。为此，我们设计了文本引导调节机制（TCM），它结合了图像区域与句子单词之间的联系，增强了捕获图像和文本之间细粒度跨模式线索的能力。该策略不仅利用了更广泛的监督来源，还为基于视觉和文本特征对齐的LLIE提供了一种新的范例。为了有效地识别并融合来自不同层级的图像和文本信息的特征，我们设计了一个信息融合注意（IFA）模块来增强不同层次的不同区域。我们将所提出的TCM和IFA整合到一个用于LLIE的自然语言监督网络中，并将其命名为NaLSuper。最后，大量的实验表明我们的提议方法NaLSuper具有强大的鲁棒性和优越的有效性。
URL
https://arxiv.org/abs/2501.06546
