Abstract
Segment Anything (SAM) has recently pushed the boundaries of segmentation by demonstrating zero-shot generalization and flexible prompting after training on over one billion masks. Despite this, its mask prediction accuracy often falls short of the precision required in real-world applications. While several refinement modules have been proposed to boost SAM's segmentation quality, achieving highly accurate object delineation within a single, unified framework remains an open challenge. Furthermore, interactive image matting, which aims to generate fine-grained alpha mattes guided by diverse user hints, has not yet been explored in the context of SAM. Insights from recent studies highlight strong correlations between segmentation and matting, suggesting the feasibility of a unified model capable of both tasks. In this paper, we introduce Segment And Matte Anything (SAMA), a lightweight extension of SAM that delivers high-quality interactive image segmentation and matting with minimal extra parameters. Our Multi-View Localization Encoder (MVLE) captures detailed features from local views, while the Localization Adapter (Local-Adapter) refines mask outputs by recovering subtle boundary details. We also incorporate two prediction heads for each task into the architecture to generate segmentation and matting masks, simultaneously. Trained on a diverse dataset aggregated from publicly available sources, SAMA achieves state-of-the-art performance across multiple segmentation and matting benchmarks, showcasing its adaptability and effectiveness in a wide range of downstream tasks.
Abstract (translated)
最近，Segment Anything模型（SAM）通过在超过十亿个掩码上训练后展示出零样本泛化和灵活提示的能力，突破了分割领域的界限。尽管如此，其掩码预测精度往往达不到实际应用所需的精确度要求。虽然已提出几种增强模块来提升SAM的分割质量，但如何在一个统一框架内实现高度准确的对象边界划分仍然是一个开放性挑战。此外，在SAM背景下尚未探索互动图像蒙版生成这一目标，旨在根据各种用户提示产生精细颗粒度的alpha蒙版。最近的研究表明，分割与蒙版生成之间存在强相关性，这暗示了可以同时执行这两项任务的统一模型的可行性。 本文介绍了Segment And Matte Anything（SAMA），这是一个SAM的轻量级扩展版本，在添加极少额外参数的同时提供高质量的互动图像分割和蒙版生成。我们的Multi-View Localization Encoder（MVLE）捕捉局部视图中的详细特征，而Localization Adapter (Local-Adapter) 通过恢复细微边界细节来精炼掩码输出。我们还在架构中为每个任务嵌入了两个预测头以同时生成分割与蒙版的掩码。 SAMA在一个从公开资源汇聚而成的多样化数据集上进行训练，在多个分割和蒙版生成基准测试中均取得了最新的性能水平，展示了其在各种下游任务中的适应性和有效性。
URL
https://arxiv.org/abs/2601.12147
