Abstract
Cross-modal misalignments, such as spatial offsets, resolution discrepancies, and semantic deficiencies, frequently occur in visible-infrared object detection (VI-OD). To mitigate this, existing methods are typically adapted into an alignment-based fusion paradigm, in which an explicit pixel- or feature-level alignment module is inserted before cross-modal fusion. However, pixel-level alignment struggles to cope with severe or mixed misalignments, whereas feature-level alignment often introduces undesirable noise into fused representations under such conditions, ultimately limiting detection performance. In this paper, we propose a novel alignment-free network (AlignFreeNet) for VI-OD. Differing from prior methods, AlignFreeNet abandons any explicit alignment and instead adopts an alignment-free fusion paradigm. Specifically, AlignFreeNet comprises two core modules: variation-guided cross-modal compensation (VCC) and frequency-guided cross-modal fusion (FCF). VCC adaptively feeds the compensated information derived from cross-modal discrepancies back into each modality, enhancing visible and infrared representations without the noise caused by explicit alignment. FCF achieves robust cross-modal fusion by suppressing task-irrelevant redundancy via frequency-domain gating, effectively mitigating noise introduced in the process. Moreover, VCC and FCF jointly exploit low- and high-frequency cues to preserve foreground contours in fused representations, effectively mitigating cross-modal blending caused by severe mixed misalignments. Extensive evaluations on DVTOD, M3FD, and DroneVehicle demonstrate that our AlignFreeNet achieves state-of-the-art performance under severe mixed misalignment conditions, highlighting its robustness and generalization.
Abstract (translated)
跨模态不对齐，如空间偏移、分辨率差异和语义缺陷，在可见光-红外目标检测（VI-OD）中频繁出现。为缓解这一问题，现有方法通常采用基于对齐的融合范式，在跨模态融合之前插入显式的像素级或特征级对齐模块。然而，像素级对齐在面对严重或多类型不对准时难以应对，而特征级对齐在这种情况下常常会向融合表示中引入不希望出现的噪声，最终限制了检测性能。本文提出了一种新颖的无对齐网络（AlignFreeNet）用于VI-OD。不同于先前的方法，AlignFreeNet放弃了任何显式的对齐操作，转而采用一种无对齐的融合范式。具体来说，AlignFreeNet包含两个核心模块：变化引导跨模态补偿（VCC）和频域导向跨模态融合（FCF）。VCC自适应地将从跨模态差异中提取到的补偿信息反馈回每一模态，增强可见光和红外表示而不会引入显式对齐导致的噪声。FCF通过在频率域中抑制任务无关冗余来实现稳健的跨模态融合，有效地减轻了过程中的噪声引入。此外，VCC和FCF共同利用低频和高频线索，在融合表示中保留前景轮廓，有效缓解了由严重混合不对齐引起的跨模态混叠问题。在DVTOD、M3FD和DroneVehicle数据集上的广泛评估表明，我们的AlignFreeNet在严重混合不对准时达到了最先进的性能，突显了其鲁棒性和泛化能力。
URL
https://arxiv.org/abs/2507.20146
