Abstract
Modern artificial neural networks, including convolutional neural networks and vision transformers, have mastered several computer vision tasks, including object recognition. However, there are many significant differences between the behavior and robustness of these systems and of the human visual system. Deep neural networks remain brittle and susceptible to many changes in the image that do not cause humans to misclassify images. Part of this different behavior may be explained by the type of features humans and deep neural networks use in vision tasks. Humans tend to classify objects according to their shape while deep neural networks seem to rely mostly on texture. Exploring this question is relevant, since it may lead to better performing neural network architectures and to a better understanding of the workings of the vision system of primates. In this work, we advance the state of the art in our understanding of this phenomenon, by extending previous analyses to a much larger set of deep neural network architectures. We found that the performance of models in image classification tasks is highly correlated with their shape bias measured at the output and penultimate layer. Furthermore, our results showed that the number of neurons that represent shape and texture are strongly anti-correlated, thus providing evidence that there is competition between these two types of features. Finally, we observed that while in general there is a correlation between performance and shape bias, there are significant variations between architecture families.
Abstract (translated)
现代人工神经网络,包括卷积神经网络和视觉转换器,已经掌握了多个计算机视觉任务,包括物体识别。然而,这些系统和人类视觉系统的行为和鲁棒性有许多显著的差异。深神经网络仍然脆性,易于在图像中发生许多变化,但这些变化不会使人类错误地分类图像。这部分不同的行为可能由人类和深神经网络在视觉任务中使用的特征类型来解释。人类通常根据形状分类对象,而深神经网络似乎主要依赖于纹理。探索这个问题是相关的,因为它可能导致更好的神经网络架构和更好地理解灵长类视觉系统的工作原理。在这项工作中,我们推动了我们对这种现象的理解的前沿,通过将先前的分析扩展到更大量的深神经网络架构。我们发现,在图像分类任务中,模型的性能与它们在输出和前导层的形状偏见高度相关。此外,我们的结果显示,代表形状和纹理的神经元数量具有很强的反相关关系,因此提供了证据,这两种特征之间存在竞争。最后,我们观察到,虽然通常在性能与形状偏见之间存在相关关系,但架构家族之间存在显著差异。
URL
https://arxiv.org/abs/2301.10608
