Abstract
We introduce ISALux, a novel transformer-based approach for Low-Light Image Enhancement (LLIE) that seamlessly integrates illumination and semantic priors. Our architecture includes an original self-attention block, Hybrid Illumination and Semantics-Aware Multi-Headed Self- Attention (HISA-MSA), which integrates illumination and semantic segmentation maps for en- hanced feature extraction. ISALux employs two self-attention modules to independently process illumination and semantic features, selectively enriching each other to regulate luminance and high- light structural variations in real-world scenarios. A Mixture of Experts (MoE)-based Feed-Forward Network (FFN) enhances contextual learning, with a gating mechanism conditionally activating the top K experts for specialized processing. To address overfitting in LLIE methods caused by distinct light patterns in benchmarking datasets, we enhance the HISA-MSA module with low-rank matrix adaptations (LoRA). Extensive qualitative and quantitative evaluations across multiple specialized datasets demonstrate that ISALux is competitive with state-of-the-art (SOTA) methods. Addition- ally, an ablation study highlights the contribution of each component in the proposed model. Code will be released upon publication.
Abstract (translated)
我们介绍了ISALux，这是一种基于变压器的新型方法，用于低光图像增强（LLIE），该方法无缝地整合了照明和语义先验知识。我们的架构包括一个独特的自注意模块——混合照明与语义感知多头自我注意力(HISA-MSA)，它结合了照明图和语义分割地图以进行增强特征提取。ISALux采用两个自注意模块分别处理照明和语义特性，通过选择性地丰富彼此来调节实际场景中的亮度和高光结构变化。基于专家混合（MoE）的前馈网络(FFN)增强了上下文学习，并使用门控机制有条件激活前K名专家进行专业化处理。 为了应对由于基准数据集中不同光照模式而导致低光图像增强方法过度拟合的问题，我们通过低秩矩阵适应(Low-rank matrix adaptation, LoRA)来改进HISA-MSA模块。在多个专业数据集上进行的广泛定性和定量评估表明，ISALux与当前最先进的(SOTA)方法具有竞争力。此外，消融研究强调了所提出模型中每个组件的作用。论文发布后将公开代码。
URL
https://arxiv.org/abs/2508.17885
