Abstract
LiveSketch is a novel algorithm for searching large image collections using hand-sketched queries. LiveSketch tackles the inherent ambiguity of sketch search by creating visual suggestions that augment the query as it is drawn, making query specification an iterative rather than one-shot process that helps disambiguate users' search intent. Our technical contributions are: a triplet convnet architecture that incorporates an RNN based variational autoencoder to search for images using vector (stroke-based) queries; real-time clustering to identify likely search intents (and so, targets within the search embedding); and the use of backpropagation from those targets to perturb the input stroke sequence, so suggesting alterations to the query in order to guide the search. We show improvements in accuracy and time-to-task over contemporary baselines using a 67M image corpus.
Abstract (translated)
LiveSketch是一种使用手绘查询搜索大型图像集合的新算法。LiveSketch通过创建可视建议来解决草图搜索固有的模糊性，在绘制时增强查询，使查询规范成为一个迭代的而不是一次性的过程，有助于消除用户搜索意图的歧义。我们的技术贡献是：一个三重convnet体系结构，它结合了一个基于RNN的变分自动编码器来使用矢量（基于笔画的）查询搜索图像；实时聚类来识别可能的搜索意图（因此，搜索嵌入中的目标）；以及使用这些目标的反向传播来干扰输入。笔画顺序，因此建议修改查询以指导搜索。我们用一个67m的图像语料库显示了在当代基线上任务的准确性和时间上的改进。
URL
https://arxiv.org/abs/1904.06611
