Abstract
Multi-label recognition (MLR) with incomplete labels is very challenging. Recent works strive to explore the image-to-label correspondence in the vision-language model, \ie, CLIP~\cite{radford2021clip}, to compensate for insufficient annotations. In spite of promising performance, they generally overlook the valuable prior about the label-to-label correspondence. In this paper, we advocate remedying the deficiency of label supervision for the MLR with incomplete labels by deriving a structured semantic prior about the label-to-label correspondence via a semantic prior prompter. We then present a novel Semantic Correspondence Prompt Network (SCPNet), which can thoroughly explore the structured semantic prior. A Prior-Enhanced Self-Supervised Learning method is further introduced to enhance the use of the prior. Comprehensive experiments and analyses on several widely used benchmark datasets show that our method significantly outperforms existing methods on all datasets, well demonstrating the effectiveness and the superiority of our method. Our code will be available at this https URL.
Abstract (translated)
不完整标签的多标签识别(MLR)是非常具有挑战性的。最近的工作致力于探索图像到标签映射在视觉语言模型中的可能性，例如\cite{radford2021clip}，以弥补缺乏标注的不足。尽管表现令人鼓舞，但他们通常忽略了关于标签到标签映射的宝贵先验。在本文中，我们倡导补救不完整标签的标签监督不足，通过从语义先验prompter中推导出结构化的语义先验来建立label-to-label映射的结构化先验。然后，我们提出了一个 novel Semantic Correspondence Prompt Network (SCPNet)，它能够全面探索结构化语义先验。此外，我们还介绍了一种增强的自监督学习方法，以增强使用先验。对多个广泛使用基准数据集的全面实验和分析表明，我们的方法和所有方法在所有数据集上显著优于现有方法，充分证明了我们方法的有效性和优越性。我们的代码将在这个 https URL上可用。
URL
https://arxiv.org/abs/2303.13223
