Abstract
Visual Question Answering (VQA) deep-learning systems tend to capture superficial statistical correlations in the training data because of strong language priors and fail to generalize to test data with a significantly different question-answer (QA) distribution. To address this issue, we introduce a self-critical training objective that ensures that visual explanations of correct answers match the most influential image regions more than other competitive answer candidates. The influential regions are either determined from human visual/textual explanations or automatically from just significant words in the question and answer. We evaluate our approach on the VQA generalization task using the VQA-CP dataset, achieving a new state-of-the-art i.e. 49.5\% using textual explanations and 48.5\% using automatically annotated regions.
Abstract (translated)
视觉问答（vqa）深度学习系统由于语言先验性强，往往会捕获训练数据中的表面统计相关性，并且无法归纳为具有显著不同问答（qa）分布的测试数据。为了解决这个问题，我们引入了一个自我批评的培训目标，确保正确答案的视觉解释比其他竞争性答案候选人更符合最具影响力的形象区域。影响区域要么由人类的视觉/文本解释决定，要么自动地由问题和答案中的重要词语决定。我们使用vqa-cp数据集评估我们对vqa泛化任务的方法，实现了一个新的最先进水平，即使用文本解释的49.5%，使用自动注释区域的48.5%。
URL
https://arxiv.org/abs/1905.09998
