Abstract
We investigate the geometric structure of learning dynamics in overparameterized transformer models through carefully controlled modular arithmetic tasks. Our primary finding is that despite operating in high-dimensional parameter spaces ($d=128$), transformer training trajectories rapidly collapse onto low-dimensional execution manifolds of dimension $3$--$4$. This dimensional collapse is robust across random seeds and moderate task difficulties, though the orientation of the manifold in parameter space varies between runs. We demonstrate that this geometric structure underlies several empirically observed phenomena: (1) sharp attention concentration emerges as saturation along routing coordinates within the execution manifold, (2) stochastic gradient descent (SGD) exhibits approximately integrable dynamics when projected onto the execution subspace, with non-integrability confined to orthogonal staging directions, and (3) sparse autoencoders capture auxiliary routing structure but fail to isolate execution itself, which remains distributed across the low-dimensional manifold. Our results suggest a unifying geometric framework for understanding transformer learning, where the vast majority of parameters serve to absorb optimization interference while core computation occurs in a dramatically reduced subspace. These findings have implications for interpretability, training curriculum design, and understanding the role of overparameterization in neural network learning.
Abstract (translated)
我们通过精心控制的模运算任务，研究了过度参数化变压器模型中学习动态的几何结构。我们的主要发现是：尽管在高维参数空间（$d=128$）内运行，变压器训练轨迹会迅速坍缩到维度为3至4的低维执行流形上。这种维度坍缩在随机种子和任务难度适中的情况下表现出稳健性，虽然不同实验中该流形在参数空间中的方向有所不同。 我们展示了这种几何结构是几种经验观察现象的基础：（1）注意力集中的突然增强表现为沿着执行流形内路由坐标上的饱和；（2）随机梯度下降（SGD）在投影到执行子空间时表现出近似可积的动力学特性，而非可积性仅限于正交阶段方向；（3）稀疏自动编码器捕捉到了辅助的路由结构，但未能隔离出执行过程本身，这仍分布在整个低维流形上。 我们的结果表明存在一个统一的几何框架来理解变压器学习，其中绝大多数参数用于吸收优化干扰，而核心计算则发生在大幅缩减的子空间中。这些发现对可解释性、训练课程设计以及过度参数化在神经网络学习中的作用的理解具有重要意义。
URL
https://arxiv.org/abs/2602.10496
