Abstract
Image fusion integrates complementary information from different modalities to generate high-quality fused images, thereby enhancing downstream tasks such as object detection and semantic segmentation. Unlike task-specific techniques that primarily focus on consolidating inter-modal information, general image fusion needs to address a wide range of tasks while improving performance without increasing complexity. To achieve this, we propose SMC-Mamba, a Self-supervised Multiplex Consensus Mamba framework for general image fusion. Specifically, the Modality-Agnostic Feature Enhancement (MAFE) module preserves fine details through adaptive gating and enhances global representations via spatial-channel and frequency-rotational scanning. The Multiplex Consensus Cross-modal Mamba (MCCM) module enables dynamic collaboration among experts, reaching a consensus to efficiently integrate complementary information from multiple modalities. The cross-modal scanning within MCCM further strengthens feature interactions across modalities, facilitating seamless integration of critical information from both sources. Additionally, we introduce a Bi-level Self-supervised Contrastive Learning Loss (BSCL), which preserves high-frequency information without increasing computational overhead while simultaneously boosting performance in downstream tasks. Extensive experiments demonstrate that our approach outperforms state-of-the-art (SOTA) image fusion algorithms in tasks such as infrared-visible, medical, multi-focus, and multi-exposure fusion, as well as downstream visual tasks.
Abstract (translated)
图像融合技术通过整合来自不同模态的互补信息，生成高质量的融合图像，从而提升下游任务如目标检测和语义分割的表现。与特定于任务的技术主要关注跨模态信息的集成不同，通用图像融合需要解决广泛的任务需求，在不增加复杂度的前提下提高性能。 为此，我们提出了SMC-Mamba框架，这是一种自监督多路径共识马玛（Self-supervised Multiplex Consensus Mamba）框架，用于通用图像融合。具体来说： - **模态不可知特征增强（Modality-Agnostic Feature Enhancement, MAFE）模块**：通过自适应门控保留精细细节，并使用空间-通道和频率旋转扫描来加强全局表示。 - **多路径共识跨模态马玛（Multiplex Consensus Cross-modal Mamba，MCCM）模块**：允许专家之间的动态协作，达成一致以高效地整合多个模态中的互补信息。MCCM内的跨模态扫描进一步增强了跨模态特征间的交互作用，促进了两种来源关键信息的无缝融合。 此外，我们还引入了双层自监督对比学习损失（Bi-level Self-supervised Contrastive Learning Loss, BSCL），该方法在不增加计算负担的情况下保留高频信息，并同时提高下游任务的表现。 通过广泛的实验表明，我们的方法在红外-可见光、医疗、多焦点和多曝光融合以及下游视觉任务等方面均超越了最先进的图像融合算法。
URL
https://arxiv.org/abs/2512.20921
