Abstract
Multimodal Image Fusion (MMIF) integrates complementary information from various modalities to produce clearer and more informative fused images. MMIF under adverse weather is particularly crucial in autonomous driving and UAV monitoring applications. However, existing adverse weather fusion methods generally only tackle single types of degradation such as haze, rain, or snow, and fail when multiple degradations coexist (e.g., haze+rain, rain+snow). To address this challenge, we propose Compound Adverse Weather Mamba (CAWM-Mamba), the first end-to-end framework that jointly performs image fusion and compound weather restoration with unified shared weights. Our network contains three key components: (1) a Weather-Aware Preprocess Module (WAPM) to enhance degraded visible features and extracts global weather embeddings; (2) a Cross-modal Feature Interaction Module (CFIM) to facilitate the alignment of heterogeneous modalities and exchange of complementary features across modalities; and (3) a Wavelet Space State Block (WSSB) that leverages wavelet-domain decomposition to decouple multi-frequency degradations. WSSB includes Freq-SSM, a module that models anisotropic high-frequency degradation without redundancy, and a unified degradation representation mechanism to further improve generalization across complex compound weather conditions. Extensive experiments on the AWMM-100K benchmark and three standard fusion datasets demonstrate that CAWM-Mamba consistently outperforms state-of-the-art methods in both compound and single-weather scenarios. In addition, our fusion results excel in downstream tasks covering semantic segmentation and object detection, confirming the practical value in real-world adverse weather perception. The source code will be available at this https URL.
Abstract (translated)
多模态图像融合（MMIF）整合了来自各种模式的互补信息，以生成更清晰、更具信息量的融合图像。在恶劣天气条件下进行的MMIF对于自动驾驶和无人机监控应用尤为关键。然而，现有的恶劣天气融合方法通常只处理单一类型的退化情况，如雾霾、雨或雪，并且当多种退化同时存在时（例如雾霾+雨，雨+雪）则无法应对。为了解决这一挑战，我们提出了复合恶劣天气蟒蛇（CAWM-Mamba），这是第一个端到端框架，它能够联合执行图像融合和复合天气恢复，采用统一的共享权重。我们的网络包含三个关键组件： 1. **气象感知预处理模块**（WAPM）：该模块旨在增强退化的可见特征，并提取全局气象嵌入。 2. **跨模态特性交互模块**（CFIM）：它促进了异构模式的对齐和跨模式之间的互补特性的交换。 3. **小波空间状态块**（WSSB）：该块利用小波域分解来分离多频退化。WSSB包括Freq-SSM，这是一个模拟各向异性高频退化的模块，并且没有冗余；以及一个统一的降解表示机制，以进一步提升在复杂复合天气条件下的泛化能力。 通过AWMM-100K基准测试和三个标准融合数据集进行的大量实验表明，在复合天气场景和单一天气场景中，CAWM-Mamba始终优于现有的最先进技术。此外，我们的融合结果在下游任务（如语义分割和目标检测）中的表现也非常出色，证实了它在实际恶劣天气感知应用中的实用价值。 源代码可在[此处](https://此链接将由您提供)获取。
URL
https://arxiv.org/abs/2603.02560
