Abstract
Marine scene understanding and segmentation plays a vital role in maritime monitoring and navigation safety. However, prevalent factors like fog and strong reflections in maritime environments cause severe image degradation, significantly compromising the stability of semantic perception. Existing restoration and enhancement methods typically target specific degradations or focus solely on visual quality, lacking end-to-end collaborative mechanisms that simultaneously improve structural recovery and semantic effectiveness. Moreover, publicly available infrared-visible datasets are predominantly collected from urban scenes, failing to capture the authentic characteristics of coupled degradations in marine environments. To address these challenges, the Infrared-Visible Maritime Ship Dataset (IVMSD) is proposed to cover various maritime scenarios under diverse weather and illumination conditions. Building upon this dataset, a Multi-task Complementary Learning Framework (MCLF) is proposed to collaboratively perform image restoration, multimodal fusion, and semantic segmentation within a unified architecture. The framework includes a Frequency-Spatial Enhancement Complementary (FSEC) module for degradation suppression and structural enhancement, a Semantic-Visual Consistency Attention (SVCA) module for semantic-consistent guidance, and a cross-modality guided attention mechanism for selective fusion. Experimental results on IVMSD demonstrate that the proposed method achieves state-of-the-art segmentation performance, significantly enhancing robustness and perceptual quality under complex maritime conditions.
Abstract (translated)
海洋场景理解与分割在海上监测和航行安全中起着至关重要的作用。然而，海上环境中常见的雾和强反射等因素会导致图像严重退化，显著影响语义感知的稳定性。现有的复原与增强方法通常针对特定退化类型，或仅关注视觉质量，缺乏能同时提升结构恢复与语义效果端到端的协同机制。此外，公开的红外-可见光数据集多采集自城市场景，未能真实反映海洋环境中耦合退化的特性。为应对这些挑战，本文提出红外-可见光海洋船舶数据集（IVMSD），涵盖不同天气与光照条件下的多种海上场景。基于该数据集，进一步提出多任务互补学习框架（MCLF），在统一架构中协同完成图像复原、多模态融合与语义分割任务。该框架包含用于退化抑制与结构增强的频空增强互补模块（FSEC）、用于语义一致引导的语义视觉一致性注意力模块（SVCA），以及用于选择性融合的跨模态引导注意力机制。在IVMSD上的实验结果表明，所提方法实现了最先进的分割性能，显著提升了复杂海况下的鲁棒性与感知质量。
URL
https://arxiv.org/abs/2603.28414
