Abstract
Work in psychology has highlighted that the geometric model of similarity standard in deep learning is not psychologically plausible because its metric properties such as symmetry do not align with human perception. In contrast, Tversky (1977) proposed an axiomatic theory of similarity based on a representation of objects as sets of features, and their similarity as a function of common and distinctive features. However, this model has not been used in deep learning before, partly due to the challenge of incorporating discrete set operations. We develop a differentiable parameterization of Tversky's similarity that is learnable through gradient descent, and derive neural network building blocks such as the Tversky projection layer, which unlike the linear projection layer can model non-linear functions such as XOR. Through experiments with image recognition and language modeling, we show that the Tversky projection layer is a beneficial replacement for the linear projection layer, which employs geometric similarity. On the NABirds image classification task, a frozen ResNet-50 adapted with a Tversky projection layer achieves a 24.7% relative accuracy improvement over the linear layer adapter baseline. With Tversky projection layers, GPT-2's perplexity on PTB decreases by 7.5%, and its parameter count by 34.8%. Finally, we propose a unified interpretation of both projection layers as computing similarities of input stimuli to learned prototypes, for which we also propose a novel visualization technique highlighting the interpretability of Tversky projection layers. Our work offers a new paradigm for thinking about the similarity model implicit in deep learning, and designing networks that are interpretable under an established theory of psychological similarity.
Abstract (translated)
心理学领域的研究指出，深度学习中基于几何模型的相似性标准在心理上是不合理的，因为其度量属性（如对称性）与人类感知不符。相比之下，Tversky（1977）提出了一种基于对象特征集表示和共同及独特特征函数的相似性公理理论。然而，由于难以将离散集合运算融入其中，这一模型以前未曾应用于深度学习中。 我们开发了Tversky相似性的可微参数化版本，使其可以通过梯度下降进行学习，并推导出神经网络构建模块（如Tversky投影层），这与传统的线性投影层不同，能够模拟非线性函数（例如XOR）。通过图像识别和语言建模的实验表明，Tversky投影层可以替代使用几何相似性的线性投影层。在NABirds图像分类任务中，一个冻结的ResNet-50模型，在添加了Tversky投影层后，相对于线性层适配基线实现了24.7%的相对准确率提升。 当使用Tversky投影层时，GPT-2在PTB上的困惑度降低了7.5%，参数量减少了34.8%。最后，我们提出了对两种投影层进行统一解释的方法，认为它们计算的是输入刺激与学习原型之间的相似性，并为此提出了一种新的可视化技术以突出Tversky投影层的可解释性。 我们的工作为深度学习中的隐含相似性模型提供了新的思考方式，并设计了在心理相似理论下具有良好可解释性的网络结构。
URL
https://arxiv.org/abs/2506.11035
