Abstract
Recent advances in medical image segmentation have been driven by deep learning; however, most existing methods remain limited by modality-specific designs and exhibit poor adaptability to dynamic medical imaging scenarios. The Segment Anything Model 2 (SAM2) and its related variants, which introduce a streaming memory mechanism for real-time video segmentation, present new opportunities for prompt-based, generalizable solutions. Nevertheless, adapting these models to medical video scenarios typically requires large-scale datasets for retraining or transfer learning, leading to high computational costs and the risk of catastrophic forgetting. To address these challenges, we propose DD-SAM2, an efficient adaptation framework for SAM2 that incorporates a Depthwise-Dilated Adapter (DD-Adapter) to enhance multi-scale feature extraction with minimal parameter overhead. This design enables effective fine-tuning of SAM2 on medical videos with limited training data. Unlike existing adapter-based methods focused solely on static images, DD-SAM2 fully exploits SAM2's streaming memory for medical video object tracking and segmentation. Comprehensive evaluations on TrackRad2025 (tumor segmentation) and EchoNet-Dynamic (left ventricle tracking) datasets demonstrate superior performance, achieving Dice scores of 0.93 and 0.97, respectively. To the best of our knowledge, this work provides an initial attempt at systematically exploring adapter-based SAM2 fine-tuning for medical video segmentation and tracking. Code, datasets, and models will be publicly available at this https URL.
Abstract (translated)
最近的医学图像分割进展主要得益于深度学习技术；然而，大多数现有的方法仍然局限于特定模态的设计，并且在动态医疗成像场景中表现出较差的适应性。引入了流式内存机制以实现实时视频分割的Segment Anything Model 2（SAM2）及其相关变体，为基于提示、可推广性的解决方案带来了新的机遇。然而，将这些模型应用于医学视频场景通常需要大规模的数据集来进行重新训练或迁移学习，这会导致高昂的计算成本和灾难性遗忘的风险。 为了应对这一挑战，我们提出了DD-SAM2，这是一个针对SAM2的有效适应框架，它结合了深度可分离膨胀适配器（DD-Adapter），以增强多尺度特征提取，并且参数开销极小。这种设计使得在训练数据有限的情况下对医学视频进行有效的微调成为可能。与现有的仅关注静态图像的基于适配器的方法不同，DD-SAM2充分利用了SAM2的流式内存机制来进行医学视频中的对象跟踪和分割。 我们在TrackRad2025（肿瘤分割）和EchoNet-Dynamic（左心室追踪）数据集上进行了全面评估。结果表明，在这两个任务中，我们的模型分别达到了Dice分数为0.93和0.97的优异性能。 据我们所知，这项工作首次系统地探索了基于适配器的SAM2微调在医学视频分割与跟踪中的应用。代码、数据集以及模型将公开发布在这个链接：[此URL]。
URL
https://arxiv.org/abs/2507.14613
