Abstract
Robust semantic segmentation of road scenes under adverse illumination, lighting, and shadow conditions remain a core challenge for autonomous driving applications. RGB-Thermal fusion is a standard approach, yet existing methods apply static fusion strategies uniformly across all conditions, allowing modality-specific noise to propagate throughout the network. Hence, we propose CLARITY that dynamically adapts its fusion strategy to the detected scene condition. Guided by vision-language model (VLM) priors, the network learns to modulate each modality's contribution based on the illumination state while leveraging object embeddings for segmentation, rather than applying a fixed fusion policy. We further introduce two mechanisms, i.e., one which preserves valid dark-object semantics that prior noise-suppression methods incorrectly discard, and a hierarchical decoder that enforces structural consistency across scales to sharpen boundaries on thin objects. Experiments on the MFNet dataset demonstrate that CLARITY establishes a new state-of-the-art (SOTA), achieving 62.3% mIoU and 77.5% mAcc.
Abstract (translated)
在恶劣光照、照明和阴影条件下对道路场景进行鲁棒的语义分割仍然是自主驾驶应用的核心挑战。RGB-热成像融合是标准方法，但现有方法采用静态融合策略，在所有情况下都统一应用，这使得特定模式下的噪声在整个网络中传播。因此，我们提出了CLARITY，该方法能够根据检测到的场景条件动态调整其融合策略。通过视觉语言模型（VLM）先验引导，网络学会基于光照状态调节每个模态的贡献，并利用对象嵌入进行分割，而不是采用固定的融合政策。 此外，我们还引入了两种机制：一种保留先前噪声抑制方法错误丢弃的有效暗物体语义；另一种是分层解码器，通过在不同尺度上强制结构一致性来锐化细小物体的边界。在MFNet数据集上的实验表明，CLARITY建立了新的最先进水平（SOTA），实现了62.3%的mIoU和77.5%的mAcc。
URL
https://arxiv.org/abs/2602.07343
