Abstract
A vexing problem in artificial intelligence is reasoning about events that occur in complex, changing visual stimuli such as in video analysis or game play. Inspired by a rich tradition of visual reasoning and memory in cognitive psychology and neuroscience, we developed an artificial, configurable visual question and answer dataset (COG) to parallel experiments in humans and animals. COG is much simpler than the general problem of video analysis, yet it addresses many of the problems relating to visual and logical reasoning and memory -- problems that remain challenging for modern deep learning architectures. We additionally propose a deep learning architecture that performs competitively on other diagnostic VQA datasets (i.e. CLEVR) as well as easy settings of the COG dataset. However, several settings of COG result in datasets that are progressively more challenging to learn. After training, the network can zero-shot generalize to many new tasks. Preliminary analyses of the network architectures trained on COG demonstrate that the network accomplishes the task in a manner interpretable to humans.
Abstract (translated)
人工智能中令人头疼的问题是推理复杂的，不断变化的视觉刺激（如视频分析或游戏）中发生的事件。受认知心理学和神经科学中视觉推理和记忆的丰富传统启发，我们开发了一种人工，可配置的视觉问题和答案数据集（COG），用于人类和动物的平行实验。 COG比视频分析的一般问题简单得多，但它解决了许多与视觉和逻辑推理和记忆相关的问题 - 对现代深度学习体系结构仍然具有挑战性的问题。我们另外提出了一种深度学习架构，该架构在其他诊断VQA数据集（即CLEVR）上具有竞争力，并且可以轻松设置COG数据集。然而，COG的几个设置会导致数据集逐渐变得更具挑战性。训练结束后，网络可以将许多新任务归零。在COG上训练的网络架构的初步分析表明，网络以人类可解释的方式完成任务。
URL
https://arxiv.org/abs/1803.06092
