Abstract
Deep Neural Networks (DNNs) often fail in out-of-distribution scenarios. In this paper, we introduce a tool to visualize and understand such failures. We draw inspiration from concepts from neural electrophysiology, which are based on inspecting the internal functioning of a neural networks by analyzing the feature tuning and invariances of individual units. Deep Electrophysiology, in short Deephys, provides insights of the DNN's failures in out-of-distribution scenarios by comparative visualization of the neural activity in in-distribution and out-of-distribution datasets. Deephys provides seamless analyses of individual neurons, individual images, and a set of set of images from a category, and it is capable of revealing failures due to the presence of spurious features and novel features. We substantiate the validity of the qualitative visualizations of Deephys thorough quantitative analyses using convolutional and transformers architectures, in several datasets and distribution shifts (namely, colored MNIST, CIFAR-10 and ImageNet).
Abstract (translated)
深度神经网络(DNN)在分布不均衡的情况下经常失败。在本文中,我们介绍了一种工具,以可视化和理解这些失败。我们借鉴了神经电生理学的概念,这些概念基于检查神经网络内部运作的分析个体单元的特征调整和不变性。深度电生理学,简称Deephys,通过比较分布不均衡数据和分布不均衡数据下的神经网络活动的对比,提供了DNN在分布不均衡情况下失败的见解。Deephys提供了 seamless 的分析单个神经元、单个图像和一组类别中的所有图像,并能够揭示由于伪特征和新特征的存在而导致的失败。我们通过卷积和转换架构的量化分析,支持了Deephys的定性可视化的效力,在多个数据集和分布转移中(例如彩色米尼汉、CIFAR-10和图像网)证明了它的定性可视化的定量可视化的有效性。
URL
https://arxiv.org/abs/2303.11912
