Abstract
We study Langevin dynamics with noise projected onto the directions orthogonal to an isometric group action. This mathematical model is introduced to shed new light on the effects of symmetry on stochastic gradient descent for over-parametrized models. Our main result identifies a novel form of implicit regularization: when the initial and target density are both invariant under the group action, Langevin dynamics with projected noise is equivalent in law to Langevin dynamics with isotropic diffusion but with an additional drift term proportional to the negative log volume of the group orbit. We prove this result by constructing a coupling of the two processes via a third process on the group itself, and identify the additional drift as the mean curvature of the orbits.
Abstract (translated)
我们研究了噪声投影到等距群作用正交方向上的兰格vin动力学。这一数学模型被引入以揭示对称性在过参数化模型的随机梯度下降中的影响。我们的主要结果确定了一种新型的隐式正则化形式：当初始密度和目标密度都在群作用下不变时，带有投影噪声的兰格vin动力学与具有额外漂移项（该漂移项与群轨道的负对数体积成比例）的各向同性扩散下的兰格vin动力学在法理上是等价的。我们通过构造两个过程之间的耦合（即在群自身上的第三个过程中），证明了这一结果，并将额外的漂移项识别为轨道的平均曲率。
URL
https://arxiv.org/abs/2602.12257
