Abstract
Accurate segmentation of polyps and skin lesions is essential for diagnosing colorectal and skin cancers. While various segmentation methods for polyps and skin lesions using fully supervised deep learning techniques have been developed, the pixel-level annotation of medical images by doctors is both time-consuming and costly. Foundational vision models like the Segment Anything Model (SAM) have demonstrated superior performance; however, directly applying SAM to medical segmentation may not yield satisfactory results due to the lack of domain-specific medical knowledge. In this paper, we propose BiSeg-SAM, a SAM-guided weakly supervised prompting and boundary refinement network for the segmentation of polyps and skin lesions. Specifically, we fine-tune SAM combined with a CNN module to learn local features. We introduce a WeakBox with two functions: automatically generating box prompts for the SAM model and using our proposed Multi-choice Mask-to-Box (MM2B) transformation for rough mask-to-box conversion, addressing the mismatch between coarse labels and precise predictions. Additionally, we apply scale consistency (SC) loss for prediction scale alignment. Our DetailRefine module enhances boundary precision and segmentation accuracy by refining coarse predictions using a limited amount of ground truth labels. This comprehensive approach enables BiSeg-SAM to achieve excellent multi-task segmentation performance. Our method demonstrates significant superiority over state-of-the-art (SOTA) methods when tested on five polyp datasets and one skin cancer dataset.
Abstract (translated)
结肠息肉和皮肤病变的精确分割对于诊断结直肠癌和皮肤癌至关重要。尽管已经开发出了多种使用全监督深度学习技术对息肉和皮肤病变进行分割的方法，但医生为医学图像提供像素级别的标注既耗时又昂贵。基础视觉模型如Segment Anything Model (SAM) 已经展示了卓越的性能；然而，直接将SAM应用于医学分割可能由于缺乏特定领域的医疗知识而无法获得令人满意的结果。在本文中，我们提出了BiSeg-SAM，这是一个由SAM引导的弱监督提示和边界细化网络，用于息肉和皮肤病变的分割。 具体来说，我们将SAM与CNN模块结合进行微调以学习局部特征。我们引入了一个名为WeakBox的功能组件：它可以自动为SAM模型生成框提示，并使用我们提出的Multi-choice Mask-to-Box (MM2B) 转换技术来进行粗略的掩码到边界框转换，解决了粗糙标签和精确预测之间的不匹配问题。此外，我们应用尺度一致性（SC）损失来对齐预测的尺度。我们的DetailRefine模块通过利用少量的真实标注数据来细化粗略的预测结果以提高边界的精度和分割准确性。这种全面的方法使BiSeg-SAM能够实现卓越的多任务分割性能。 在五个息肉数据集和一个皮肤癌数据集中进行测试时，我们的方法相对于最先进的（SOTA）方法表现出显著的优势。
URL
https://arxiv.org/abs/2504.01452
