Abstract
Existing question answering methods often assume that the input content (e.g., documents or videos) is always accessible to solve the task. Alternatively, memory networks were introduced to mimic the human process of incremental comprehension and compression of the information in a fixed-capacity memory. However, these models only learn how to maintain memory by backpropagating errors in the answers through the entire network. Instead, it has been suggested that humans have effective mechanisms to boost their memorization capacities, such as rehearsal and anticipation. Drawing inspiration from these, we propose a memory model that performs rehearsal and anticipation while processing inputs to memorize important information for solving question answering tasks from streaming data. The proposed mechanisms are applied self-supervised during training through masked modeling tasks focused on coreference information. We validate our model on a short-sequence (bAbI) dataset as well as large-sequence textual (NarrativeQA) and video (ActivityNet-QA) question answering datasets, where it achieves substantial improvements over previous memory network approaches. Furthermore, our ablation study confirms the proposed mechanisms' importance for memory models.
Abstract (translated)
现有的回答方法通常假设输入内容(例如文档或视频)总是可用来解决问题。Alternatively，引入记忆网络是为了模拟人类在固定容量内存中逐渐增加理解和压缩信息的过程。然而，这些模型只会通过学习整个网络中答案的错误反向传播来维护记忆。相反，有人建议人类有有效的机制来增强记忆能力，例如复习和预测。从这些中汲取灵感，我们提出了一个记忆模型，在处理输入时进行复习和预测，以从流数据中解决问答任务并记忆重要的信息。在训练过程中，我们使用基于掩码关联信息的任务进行自监督训练。我们验证了我们模型在短序列(bAbI)数据集和大型文本(NarrativeQA)和视频(ActivityNet-QA)问答数据集上的性能，比先前的记忆网络方法取得了显著的改进。此外，我们的去除了研究确认了所提出的机制对于记忆模型的重要性。
URL
https://arxiv.org/abs/2305.07565
