Abstract
Segment Anything Model (SAM), known for its remarkable zero-shot segmentation capabilities, has garnered significant attention in the community. Nevertheless, its performance is challenged when dealing with what we refer to as visually non-salient scenarios, where there is low contrast between the foreground and background. In these cases, existing methods often cannot capture accurate contours and fail to produce promising segmentation results. In this paper, we propose Visually Non-Salient SAM (VNS-SAM), aiming to enhance SAM's perception of visually non-salient scenarios while preserving its original zero-shot generalizability. We achieve this by effectively exploiting SAM's low-level features through two designs: Mask-Edge Token Interactive decoder and Non-Salient Feature Mining module. These designs help the SAM decoder gain a deeper understanding of non-salient characteristics with only marginal parameter increments and computational requirements. The additional parameters of VNS-SAM can be optimized within 4 hours, demonstrating its feasibility and practicality. In terms of data, we established VNS-SEG, a unified dataset for various VNS scenarios, with more than 35K images, in contrast to previous single-task adaptations. It is designed to make the model learn more robust VNS features and comprehensively benchmark the model's segmentation performance and generalizability on VNS scenarios. Extensive experiments across various VNS segmentation tasks demonstrate the superior performance of VNS-SAM, particularly under zero-shot settings, highlighting its potential for broad real-world applications. Codes and datasets are publicly available at this https URL.
Abstract (translated)
段落翻译如下： Segment Anything Model (SAM)，因其卓越的零样本分割能力而闻名，已经在社区中引起了广泛关注。然而，在我们称之为视觉非显著场景（即前景和背景对比度较低）的情况下，其性能受到挑战。在这些情况下，现有的方法往往无法准确捕捉轮廓，并且难以生成有希望的分割结果。在本文中，我们提出了Visually Non-Salient SAM (VNS-SAM)，旨在增强SAM对视觉非显著场景的理解能力，同时保持其原有的零样本泛化能力。通过有效利用SAM的低级特征，我们在Mask-Edge Token Interactive解码器和Non-Salient Feature Mining模块的设计上实现了这一点。这些设计帮助SAM解码器在参数增量和计算需求仅略有增加的情况下更深入地理解非显著特性。VNS-SAM中的额外参数可以在4小时内优化完成，这证明了其可行性和实用性。 在数据方面，我们建立了VNS-SEG，这是一个统一的、针对各种视觉非显著场景的数据集，包含超过35,000张图像，与之前的单一任务适应相比具有明显优势。该数据集旨在使模型学习更稳健的视觉非显著特性，并全面评估模型在视觉非显著场景中的分割性能和泛化能力。通过多种视觉非显著分割任务进行广泛的实验表明，在零样本设置下VNS-SAM表现出色，突显了其在广泛现实世界应用中的潜力。代码和数据集公开可用，请访问此链接：[提供链接]。 请注意，原文的最后提供了代码和数据集的访问链接，您需要根据实际情况填写具体URL地址以供读者参考。
URL
https://arxiv.org/abs/2601.00537
