Abstract
Traditional image recognition methods only consider objects belonging to already learned classes. However, since training a recognition model with every object class in the world is unfeasible, a way of getting information on unknown objects (i.e., objects whose class has not been learned) is necessary. A way for an image recognition system to learn new classes could be asking a human about objects that are unknown. In this paper, we propose a method for generating questions about unknown objects in an image, as means to get information about classes that have not been learned. Our method consists of a module for proposing objects, a module for identifying unknown objects, and a module for generating questions about unknown objects. The experimental results via human evaluation show that our method can successfully get information about unknown objects in an image dataset. Our code and dataset are available at https://github.com/mil-tokyo/vqg-unknown.
Abstract (translated)
传统的图像识别方法仅考虑属于已学习的类的对象。然而，由于训练世界上每个对象类的识别模型是不可行的，因此需要一种获取未知对象（即，其类尚未学习的对象）的信息的方法。图像识别系统学习新类的方法可能是向人类询问未知的对象。在本文中，我们提出了一种方法，用于生成有关图像中未知对象的问题，作为获取有关尚未学习的类的信息的方法。我们的方法包括一个用于提出对象的模块，一个用于识别未知对象的模块，以及一个用于生成有关未知对象的问题的模块。通过人工评估的实验结果表明，我们的方法可以成功地获得图像数据集中有关未知对象的信息。我们的代码和数据集可在https://github.com/mil-tokyo/vqg-unknown上找到。
URL
https://arxiv.org/abs/1808.01821
