Abstract
Sketch-based image retrieval (SBIR) is widely recognized as an important vision problem which implies a wide range of real-world applications. Recently, research interests arise in solving this problem under the more realistic and challenging setting of zero-shot learning. In this paper, we investigate this problem from the viewpoint of domain adaptation which we show is critical in improving feature embedding in the zero-shot scenario. Based on a framework which starts with a pre-trained model on ImageNet and fine-tunes it on the training set of SBIR benchmark, we advocate the importance of preserving previously acquired knowledge, e.g., the rich discriminative features learned from ImageNet, so as to improve the model's transfer ability. For this purpose, we design an approach named Semantic-Aware Knowledge prEservation (SAKE), which fine-tunes the pre-trained model in an economical way and leverages semantic information, e.g., inter-class relationship, to achieve the goal of knowledge preservation. Zero-shot experiments on two extended SBIR datasets, TU-Berlin and Sketchy, verify the superior performance of our approach. Extensive diagnostic experiments validate that knowledge preserved benefits SBIR in zero-shot settings, as a large fraction of the performance gain is from the more properly structured feature embedding for photo images.
Abstract (translated)
基于草图的图像检索是一个重要的视觉问题，具有广泛的现实应用前景。近年来，在更现实、更具挑战性的零镜头学习环境下，解决这一问题的研究兴趣不断涌现。在本文中，我们从域自适应的角度来研究这个问题，这对于改善零镜头场景中的特征嵌入是至关重要的。基于一个以图像网络预训练模型为基础，在SBIR基准训练集上对其进行微调的框架，我们主张保留已有知识的重要性，如从图像网络学习到的丰富的识别特征，以提高模型的传递能力。为此，我们设计了一种称为语义感知知识保存的方法，它以经济的方式微调预先训练的模型，并利用语义信息，如阶级间的关系，以达到知识保存的目的。对两个扩展的SBIR数据集Tu Berlin和Sketchy进行了零镜头实验，验证了我们方法的优越性能。大量的诊断实验验证了知识保留在零镜头设置下对SBIR的好处，因为大部分性能增益来自于照片图像的更合理的结构特征嵌入。
URL
https://arxiv.org/abs/1904.03208
