Abstract
Accurate automatic medical image segmentation relies on high-quality, dense annotations, which are costly and time-consuming. Weakly supervised learning provides a more efficient alternative by leveraging sparse and coarse annotations instead of dense, precise ones. However, segmentation performance degradation and overfitting caused by sparse annotations remain key challenges. To address these issues, we propose CmFNet, a novel 3D weakly supervised cross-modal medical image segmentation approach. CmFNet consists of three main components: a modality-specific feature learning network, a cross-modal feature learning network, and a hybrid-supervised learning strategy. Specifically, the modality-specific feature learning network and the cross-modal feature learning network effectively integrate complementary information from multi-modal images, enhancing shared features across modalities to improve segmentation performance. Additionally, the hybrid-supervised learning strategy guides segmentation through scribble supervision, intra-modal regularization, and inter-modal consistency, modeling spatial and contextual relationships while promoting feature alignment. Our approach effectively mitigates overfitting, delivering robust segmentation results. It excels in segmenting both challenging small tumor regions and common anatomical structures. Extensive experiments on a clinical cross-modal nasopharyngeal carcinoma (NPC) dataset (including CT and MR imaging) and the publicly available CT Whole Abdominal Organ dataset (WORD) show that our approach outperforms state-of-the-art weakly supervised methods. In addition, our approach also outperforms fully supervised methods when full annotation is used. Our approach can facilitate clinical therapy and benefit various specialists, including physicists, radiologists, pathologists, and oncologists.
Abstract (translated)
准确的自动医学图像分割依赖于高质量、密集的标注，这既耗时又昂贵。弱监督学习通过利用稀疏和粗略的标注提供了一种更高效的替代方案，而不是使用密集且精确的标注。然而，由于稀疏标注导致的分割性能下降以及过拟合问题仍然是关键挑战。为了解决这些问题，我们提出了CmFNet——一种新颖的3D弱监督跨模态医学图像分割方法。CmFNet由三个主要组成部分组成：特定模态特征学习网络、跨模态特征学习网络和混合监督学习策略。 具体来说，特定模态特征学习网络和跨模态特征学习网络有效整合了多模态图像中的互补信息，增强各模态之间的共享特性，从而提高分割性能。此外，混合监督学习策略通过涂鸦监督（scribble supervision）、同模态正则化（intra-modal regularization）以及异模态一致性（inter-modal consistency），建模空间和上下文关系，并促进特征对齐。我们的方法有效地缓解了过拟合问题，提供了鲁棒的分割结果。 它在分割具有挑战性的微小肿瘤区域和常见解剖结构方面表现出色。通过临床跨模态鼻咽癌（NPC）数据集（包括CT和MR成像）以及公开可用的CT全腹部器官数据集（WORD）进行广泛的实验表明，我们的方法优于现有的弱监督方法。此外，在使用完整标注时，我们的方法也超越了完全监督的方法。 该方法可以促进临床治疗，并有利于物理学家、放射科医生、病理学家和肿瘤学家等各专业人员。
URL
https://arxiv.org/abs/2506.18042
