Abstract
Interpretability of deep neural networks (DNNs) is essential since it enables users to understand the overall strengths and weaknesses of the models, conveys an understanding of how the models will behave in the future, and how to diagnose and correct potential problems. However, it is challenging to reason about what a DNN actually does due to its opaque or black-box nature. To address this issue, we propose a novel technique to improve the interpretability of DNNs by leveraging the rich semantic information embedded in human descriptions. By concentrating on the video captioning task, we first extract a set of semantically meaningful topics from the human descriptions that cover a wide range of visual concepts, and integrate them into the model with an interpretive loss. We then propose a prediction difference maximization algorithm to interpret the learned features of each neuron. Experimental results demonstrate its effectiveness in video captioning using the interpretable features, which can also be transferred to video action recognition. By clearly understanding the learned features, users can easily revise false predictions via a human-in-the-loop procedure.
Abstract (translated)
深度神经网络（DNNs）的可解释性是至关重要的，因为它使用户能够了解模型的整体优势和弱点，传达对模型将来如何表现的理解，以及如何诊断和纠正潜在问题。然而，由于它的不透明或黑盒性质，推断DNN实际做了什么是具有挑战性的。为了解决这个问题，我们提出了一种新颖的技术，通过利用嵌入在人类描述中的丰富的语义信息来提高DNN的可解释性。通过专注于视频字幕任务，我们首先从涵盖大范围视觉概念的人类描述中提取一组语义上有意义的话题，并将它们整合到模型中并带有解释性损失。然后，我们提出一个预测差分最大化算法来解释每个神经元的学习特征。实验结果证明了它在使用可解释特征的视频字幕中的有效性，其也可以被转换为视频动作识别。通过清楚了解学习到的特征，用户可以通过人在回路过程轻松修改错误预测。
URL
https://arxiv.org/abs/1703.04096
