Abstract
Foundation segmentation models such as the Segment Anything Model (SAM) exhibit strong zero-shot generalization through large-scale pretraining, but adapting them to domain-specific semantic segmentation remains challenging, particularly for thin structures (e.g., retinal vessels) and noisy modalities (e.g., SAR imagery). Full fine-tuning is computationally expensive and risks catastrophic forgetting. We propose \textbf{TopoLoRA-SAM}, a topology-aware and parameter-efficient adaptation framework for binary semantic segmentation. TopoLoRA-SAM injects Low-Rank Adaptation (LoRA) into the frozen ViT encoder, augmented with a lightweight spatial convolutional adapter and optional topology-aware supervision via differentiable clDice. We evaluate our approach on five benchmarks spanning retinal vessel segmentation (DRIVE, STARE, CHASE\_DB1), polyp segmentation (Kvasir-SEG), and SAR sea/land segmentation (SL-SSDD), comparing against U-Net, DeepLabV3+, SegFormer, and Mask2Former. TopoLoRA-SAM achieves the best retina-average Dice and the best overall average Dice across datasets, while training only \textbf{5.2\%} of model parameters ($\sim$4.9M). On the challenging CHASE\_DB1 dataset, our method substantially improves segmentation accuracy and robustness, demonstrating that topology-aware parameter-efficient adaptation can match or exceed fully fine-tuned specialist models. Code is available at : this https URL
Abstract (translated)
基于大范围预训练的分割模型（如Segment Anything Model，SAM）展现出强大的零样本泛化能力。然而，将这些基础模型调整为特定领域的语义分割任务仍然具有挑战性，尤其是在处理细长结构（例如视网膜血管）和噪声模式（例如SAR图像）时更为困难。全量微调在计算上是昂贵的，并且存在灾难性遗忘的风险。 我们提出了一种基于拓扑感知并参数高效的适应框架——TopoLoRA-SAM，用于二值语义分割任务。该方法向冻结的ViT编码器中注入低秩适配（Low-Rank Adaptation, LoRA），结合了一个轻量级的空间卷积适配器，并可选地通过不同的clDice进行拓扑感知监督。 我们在五个基准数据集上进行了评估，这些数据集涵盖了视网膜血管分割（DRIVE、STARE和CHASE_DB1）、息肉分割（Kvasir-SEG）以及SAR海陆分割（SL-SSDD），并与U-Net、DeepLabV3+、SegFormer及Mask2Former等方法进行了比较。TopoLoRA-SAM在所有数据集上获得了最高的视网膜平均Dice值和总体平均Dice值，同时仅训练了模型参数的5.2%（约490万）。 特别是在具有挑战性的CHASE_DB1数据集上，我们的方法显著提高了分割准确性和鲁棒性，证明了拓扑感知且参数高效的适应可以匹配或超过全量微调的专业模型。代码可在提供的链接处获取：this https URL
URL
https://arxiv.org/abs/2601.02273
