Abstract
There has long been a belief that high-level semantics learning can benefit various downstream computer vision tasks. However, in the low-light image enhancement (LLIE) community, existing methods learn a brutal mapping between low-light and normal-light domains without considering the semantic information of different regions, especially in those extremely dark regions that suffer from severe information loss. To address this issue, we propose a new deep semantic prior-guided framework (DeepSPG) based on Retinex image decomposition for LLIE to explore informative semantic knowledge via a pre-trained semantic segmentation model and multimodal learning. Notably, we incorporate both image-level semantic prior and text-level semantic prior and thus formulate a multimodal learning framework with combinatorial deep semantic prior guidance for LLIE. Specifically, we incorporate semantic knowledge to guide the enhancement process via three designs: an image-level semantic prior guidance by leveraging hierarchical semantic features from a pre-trained semantic segmentation model; a text-level semantic prior guidance by integrating natural language semantic constraints via a pre-trained vision-language model; a multi-scale semantic-aware structure that facilitates effective semantic feature incorporation. Eventually, our proposed DeepSPG demonstrates superior performance compared to state-of-the-art methods across five benchmark datasets. The implementation details and code are publicly available at this https URL.
Abstract (translated)
长期以来，人们认为高层次的语义学习能够对各种下游计算机视觉任务产生积极影响。然而，在低光图像增强（LLIE）领域，现有的方法往往通过粗暴地映射低光和正常光照域来工作，并未考虑不同区域的语义信息，尤其是在那些严重信息损失的极暗区域内。为了解决这个问题，我们提出了一种新的基于Retinex图像分解的深度语义先验引导框架（DeepSPG），利用预训练的语义分割模型和多模态学习探索有用的语义知识。值得注意的是，我们的方法同时整合了图像级别的语义先验和文本级别的语义先验，并由此构建了一个结合组合式深度语义先验指导的多模态学习框架来处理LLIE问题。 具体而言，我们通过三种设计将语义知识融入增强过程：一种是利用预训练的语义分割模型中的层次化语义特征提供的图像级别语义先验引导；另一种是由预训练的视觉-语言模型整合自然语言语义约束构成的文本级别语义先验引导；还有一种是多尺度感知结构，这有助于有效结合语义特征。最终，在五个基准数据集上与现有最优方法相比，我们提出的DeepSPG框架表现出了更优性能。 有关实现细节和代码，请访问：[此处提供链接]（请将此占位符替换为实际的网址）。
URL
https://arxiv.org/abs/2504.19127
