Abstract
Humans and other animals readily generalize abstract relations, such as recognizing constant in shape or color, whereas neural networks struggle. To investigate how neural networks generalize abstract relations, we introduce SimplifiedRPM, a novel benchmark for systematic evaluation. In parallel, we conduct human experiments to benchmark relational difficulty, enabling direct model-human comparisons. Testing four architectures--ResNet-50, Vision Transformer, Wild Relation Network, and Scattering Compositional Learner (SCL)--we find that SCL best aligns with human behavior and generalizes best. Building on a geometric theory of neural representations, we show representational geometries that predict generalization. Layer-wise analysis reveals distinct relational reasoning strategies across models and suggests a trade-off where unseen rule representations compress into training-shaped subspaces. Guided by our geometric perspective, we propose and evaluate SNRloss, a novel objective balancing representation geometry. Our findings offer geometric insights into how neural networks generalize abstract relations, paving the way for more human-like visual reasoning in AI.
Abstract (translated)
人类和其他动物能够轻松地概括抽象关系，例如识别形状或颜色不变性，而神经网络在这方面却表现得较为困难。为了研究神经网络如何概括抽象关系，我们引入了SimplifiedRPM这一新的基准测试平台，用于系统的评估。同时，我们也进行了人类实验以衡量关系的难度，并直接比较模型与人的行为。 通过对四种架构——ResNet-50、Vision Transformer、Wild Relation Network 和 Scattering Compositional Learner（SCL）进行测试，我们发现SCL在与人类行为对齐以及泛化能力方面表现最佳。基于神经表示的几何理论，我们展示了能够预测概括效果的表示几何结构，并且通过对各个层级的分析揭示了不同模型之间迥异的关系推理策略，同时表明未见过的规则表示会在训练形状子空间中压缩。 根据我们的几何视角，我们提出了并评估了一个新的目标函数SNRloss，它平衡了表示几何。这一发现为神经网络如何概括抽象关系提供了几何学见解，并为进一步在AI领域实现更接近人类视觉推理的能力开辟了道路。
URL
https://arxiv.org/abs/2502.17382
