Abstract
Unsupervised instance segmentation aims to segment distinct object instances in an image without relying on human-labeled data. This field has recently seen significant advancements, partly due to the strong local correspondences afforded by rich visual feature representations from self-supervised models (e.g., DINO). Recent state-of-the-art approaches use self-supervised features to represent images as graphs and solve a generalized eigenvalue system (i.e., normalized-cut) to generate foreground masks. While effective, this strategy is limited by its attendant computational demands, leading to slow inference speeds. In this paper, we propose Prompt and Merge (ProMerge), which leverages self-supervised visual features to obtain initial groupings of patches and applies a strategic merging to these segments, aided by a sophisticated background-based mask pruning technique. ProMerge not only yields competitive results but also offers a significant reduction in inference time compared to state-of-the-art normalized-cut-based approaches. Furthermore, when training an object detector using our mask predictions as pseudo-labels, the resulting detector surpasses the current leading unsupervised model on various challenging instance segmentation benchmarks.
Abstract (translated)
无监督实例分割旨在在没有依赖人类标注数据的情况下分割图像中的独特物体实例。这一领域最近取得了显著的进展，部分得益于来自自监督模型（如DINO）的丰富视觉特征表示。最近的最先进方法使用自监督特征将图像表示为图，并解决一个一般特征值系统（即归一化切）以生成前景掩码。虽然这种策略有效，但它的计算需求导致其速度较慢。在本文中，我们提出Prompt and Merge（ProMerge），它利用自监督视觉特征获得初始聚类，并利用一种先进的基于背景的掩码修剪技术对聚类进行战略合并。ProMerge不仅产生了竞争力的结果，而且与基于归一化切的状态最先进方法相比，显著减少了推理时间。此外，当使用我们掩码预测作为伪标签训练物体检测器时，训练出的检测器在各种具有挑战性的实例分割基准测试中超过了当前领先的无监督模型。
URL
https://arxiv.org/abs/2409.18961
