Abstract
Interactive segmentation enables users to segment as needed by providing cues of objects, which introduces human-computer interaction for many fields, such as image editing and medical image analysis. Typically, massive and expansive pixel-level annotations are spent to train deep models by object-oriented interactions with manually labeled object masks. In this work, we reveal that informative interactions can be made by simulation with semantic-consistent yet diverse region exploration in an unsupervised paradigm. Concretely, we introduce a Multi-granularity Interaction Simulation (MIS) approach to open up a promising direction for unsupervised interactive segmentation. Drawing on the high-quality dense features produced by recent self-supervised models, we propose to gradually merge patches or regions with similar features to form more extensive regions and thus, every merged region serves as a semantic-meaningful multi-granularity proposal. By randomly sampling these proposals and simulating possible interactions based on them, we provide meaningful interaction at multiple granularities to teach the model to understand interactions. Our MIS significantly outperforms non-deep learning unsupervised methods and is even comparable with some previous deep-supervised methods without any annotation.
Abstract (translated)
交互式分割通过提供对象的线索，使用户根据需要进行分割，在许多领域，如图像编辑和医学影像分析，引入了人类-计算机互动。通常情况下，大量的、昂贵的像素级注释用于训练深度模型，通过对象导向的互动与手动标注的对象 masks 进行。在这项工作中，我们揭示了通过无监督范式中模拟语义 consistent 且多样性丰富的区域，可以实现 informative 的互动。具体来说，我们介绍了一种多粒度互动模拟(MIS)方法，以打开无监督交互分割的一个有前途的方向。基于最近自监督模型产生的高质量密集特征，我们提议逐渐合并具有相似特征的补丁或区域，以形成更广泛的区域，因此，每个合并区域都是语义有意义的多粒度建议。通过随机采样这些建议并基于它们模拟可能的互动，我们提供了有意义的多粒度交互，以教模型理解互动。我们的 MIS 显著超越了非深度学习无监督方法，甚至与一些没有标注的先前深度监督方法相当。
URL
https://arxiv.org/abs/2303.13399
