Abstract
Despite the great success of the deep features in content-based image retrieval, the visual instance search remains challenging due to the lack of effective instance level feature representation. Supervised or weakly supervised object detection methods are not among the options due to their poor performance on the unknown object categories. In this paper, based on the feature set output from self-supervised ViT, the instance level region discovery is modeled as detecting the compact feature subsets in a hierarchical fashion. The hierarchical decomposition results in a hierarchy of feature subsets. The non-leaf nodes and leaf nodes on the hierarchy correspond to the various instance regions in an image of different semantic scales. The hierarchical decomposition well addresses the problem of object embedding and occlusions, which are widely observed in the real scenarios. The features derived from the nodes on the hierarchy make up a comprehensive representation for the latent instances in the image. Our instance-level descriptor remains effective on both the known and unknown object categories. Empirical studies on three instance search benchmarks show that it outperforms state-of-the-art methods considerably.
Abstract (translated)
尽管基于内容的图像检索中的深度特征取得了巨大成功，但由于缺乏有效的实例级别特征表示，视觉实例搜索仍然具有挑战性。由于在未知对象类别上的表现不佳，监督或弱监督的对象检测方法并不适用。本文基于自监督ViT输出的特征集，将实例级别的区域发现建模为以分层方式检测紧凑的特征子集的过程。这种层次化分解产生了一系列特征子集层级结构。该层级结构中的非叶节点和叶节点对应于图像中不同语义尺度的各种实例区域。这种层次化分解很好地解决了对象嵌入和遮挡的问题，这些问题在现实场景中普遍存在。从这些层级上的节点派生的特征构成了图像中潜在实例的整体表示。我们的实例级别描述符对已知和未知的对象类别都保持有效。在三个实例搜索基准数据集上的实证研究表明，它显著优于现有方法。
URL
https://arxiv.org/abs/2506.16745
