Abstract
Deep learning-based image enhancement methods show significant advantages in reducing noise and improving visibility in low-light conditions. These methods are typically based on one-to-one mapping, where the model learns a direct transformation from low light to specific enhanced images. Therefore, these methods are inflexible as they do not allow highly personalized mapping, even though an individual's lighting preferences are inherently personalized. To overcome these limitations, we propose a new light enhancement task and a new framework that provides customized lighting control through prompt-driven, semantic-level, and quantitative brightness adjustments. The framework begins by leveraging a Large Language Model (LLM) to understand natural language prompts, enabling it to identify target objects for brightness adjustments. To localize these target objects, the Retinex-based Reasoning Segment (RRS) module generates precise target localization masks using reflection images. Subsequently, the Text-based Brightness Controllable (TBC) module adjusts brightness levels based on the generated illumination map. Finally, an Adaptive Contextual Compensation (ACC) module integrates multi-modal inputs and controls a conditional diffusion model to adjust the lighting, ensuring seamless and precise enhancements accurately. Experimental results on benchmark datasets demonstrate our framework's superior performance at increasing visibility, maintaining natural color balance, and amplifying fine details without creating artifacts. Furthermore, its robust generalization capabilities enable complex semantic-level lighting adjustments in diverse open-world environments through natural language interactions.
Abstract (translated)
基于深度学习的图像增强方法在减少噪声和提高低光条件下的可视性方面表现出显著优势。这些方法通常依赖于一对一映射，即模型从低光照直接转换到特定的增强图像上进行学习。因此，这类方法不够灵活，无法提供高度个性化的映射调整，尽管每个人的照明偏好是独特的。为了克服这一限制，我们提出了一项新的光增强任务和一个新框架，该框架能够通过提示驱动、语义级别以及亮度量化的自定义光照控制来实现个性化。 该框架首先利用大型语言模型（LLM）理解自然语言指令，并识别需要进行亮度调整的目标对象。为了精确定位这些目标对象，基于Retinex的推理分割(RRS)模块使用反射图像生成精确的目标定位掩码。接着，文本驱动亮度可控(TBC)模块根据生成的照明图来调节亮度水平。最后，自适应上下文补偿(ACC)模块结合多模态输入，并控制条件扩散模型以调整光照，从而确保无缝且精准的增强效果。 在基准数据集上的实验结果显示，我们的框架在提高可见性、保持自然色彩平衡以及放大细节点方面优于现有技术，同时不产生伪影。此外，其强大的泛化能力使其能够通过自然语言互动，在多样的开放世界环境中实现复杂的语义级光照调整。
URL
https://arxiv.org/abs/2503.08168
