Abstract
We study the issue of catastrophic forgetting in the context of neural multimodal approaches to Visual Question Answering (VQA). Motivated by evidence from psycholinguistics, we devise a set of linguistically-informed VQA tasks, which differ by the types of questions involved (Wh-questions and polar questions). We test what impact task difficulty has on continual learning, and whether the order in which a child acquires question types facilitates computational models. Our results show that dramatic forgetting is at play and that task difficulty and order matter. Two well-known current continual learning methods mitigate the problem only to a limiting degree.
Abstract (translated)
我们研究了在视觉问题回答（VQA）的神经多模方法背景下的灾难性遗忘问题。在心理语言学证据的激励下，我们设计了一套语言认知的vqa任务，这些任务因涉及的问题类型（wh问题和polar问题）不同而有所不同。我们测试任务难度对持续学习有什么影响，以及孩子获得问题类型的顺序是否有助于计算模型。我们的研究结果表明，戏剧性的遗忘是在发挥作用的，任务的难度和秩序很重要。目前已知的两种连续学习方法只能在一定程度上缓解问题。
URL
https://arxiv.org/abs/1906.04229
