Gradient_Descent
-
The Geometry of Grokking: Norm Minimization on the Zero-Loss Manifold
2026-01-08 19:35:20Tiberiu MusatAbstract
Grokking is a puzzling phenomenon in neural networks where full generalization occurs only after a substantial delay following the complete memorization of the training data. Previous research has linked this delayed generalization to representation learning driven by weight decay, but the precise underlying dynamics remain elusive. In this paper, we argue that post-memorization learning can be understood through the lens of constrained optimization: gradient descent effectively minimizes the weight norm on the zero-loss manifold. We formally prove this in the limit of infinitesimally small learning rates and weight decay coefficients. To further dissect this regime, we introduce an approximation that decouples the learning dynamics of a subset of parameters from the rest of the network. Applying this framework, we derive a closed-form expression for the post-memorization dynamics of the first layer in a two-layer network. Experiments confirm that simulating the training process using our predicted gradients reproduces both the delayed generalization and representation learning characteristic of grokking.
Abstract (translated)
Grokking 是神经网络中的一种令人困惑的现象，即在完全记住训练数据之后，会出现一段延迟期，在此期间不会发生泛化，直到某个时刻才突然实现全面的泛化。先前的研究已经将这种延迟泛化与受权重衰减驱动的表现学习联系起来，但其背后的确切动态机制仍然不清楚。在这篇论文中，我们主张可以通过约束优化的视角来理解记忆后学习：在零损失流形上，梯度下降有效最小化了权重范数。我们在极限条件下（即学习率和权重衰减系数无限小的情况下）正式证明了这一点。为了进一步剖析这一机制，我们引入了一种近似方法，将一部分参数的学习动态与网络其余部分解耦。通过应用这个框架，我们推导出了双层网络中第一层在记忆后阶段的封闭形式表达式。实验结果证实，使用我们的预测梯度模拟训练过程可以再现 grokking 的延迟泛化和表示学习特性。 简单来说，这篇论文提出了一种理解神经网络中 Grokking 现象的新方法，即通过约束优化来解析权重衰减对模型泛化能力的影响，并且理论推导与实验结果吻合良好。
URL
https://arxiv.org/abs/2511.01938
PDF
-
Parameter-Robust MPPI for Safe Online Learning of Unknown Parameters
2026-01-06 11:44:11Matti Vahs, Jaeyoun Choi, Niklas Schmid, Jana Tumova, Chuchu FanAbstract
Robots deployed in dynamic environments must remain safe even when key physical parameters are uncertain or change over time. We propose Parameter-Robust Model Predictive Path Integral (PRMPPI) control, a framework that integrates online parameter learning with probabilistic safety constraints. PRMPPI maintains a particle-based belief over parameters via Stein Variational Gradient Descent, evaluates safety constraints using Conformal Prediction, and optimizes both a nominal performance-driven and a safety-focused backup trajectory in parallel. This yields a controller that is cautious at first, improves performance as parameters are learned, and ensures safety throughout. Simulation and hardware experiments demonstrate higher success rates, lower tracking error, and more accurate parameter estimates than baselines.
Abstract (translated)
在动态环境中部署的机器人必须即使在关键物理参数不确定或随时间变化的情况下，也能保持安全。为此，我们提出了一个名为Parameter-Robust Model Predictive Path Integral (PRMPPI)控制的框架，该框架将在线参数学习与概率安全性约束相结合。PRMPPI通过Stein变分梯度下降法维护基于粒子的参数信念，使用符合预测来评估安全性约束，并且同时优化名义上的性能驱动轨迹和一个专注于安全性的备用轨迹。这会产生一种控制器，在一开始时保持谨慎，随着参数的学习而提高性能，并在整个过程中确保安全性。 模拟实验与硬件实验证明，相比基准方法，PRMPPI实现了更高的成功率、更低的跟踪误差以及更准确的参数估计。
URL
https://arxiv.org/abs/2601.02948
PDF
-
Topology-Independent Robustness of the Weighted Mean under Label Poisoning Attacks in Heterogeneous Decentralized Learning
2026-01-06 03:32:34Jie Peng, Weiyu Li, Stefan Vlaski, Qing LingAbstract
Robustness to malicious attacks is crucial for practical decentralized signal processing and machine learning systems. A typical example of such attacks is label poisoning, meaning that some agents possess corrupted local labels and share models trained on these poisoned data. To defend against malicious attacks, existing works often focus on designing robust aggregators; meanwhile, the weighted mean aggregator is typically considered a simple, vulnerable baseline. This paper analyzes the robustness of decentralized gradient descent under label poisoning attacks, considering both robust and weighted mean aggregators. Theoretical results reveal that the learning errors of robust aggregators depend on the network topology, whereas the performance of weighted mean aggregator is topology-independent. Remarkably, the weighted mean aggregator, although often considered vulnerable, can outperform robust aggregators under sufficient heterogeneity, particularly when: (i) the global contamination rate (i.e., the fraction of poisoned agents for the entire network) is smaller than the local contamination rate (i.e., the maximal fraction of poisoned neighbors for the regular agents); (ii) the network of regular agents is disconnected; or (iii) the network of regular agents is sparse and the local contamination rate is high. Empirical results support our theoretical findings, highlighting the important role of network topology in the robustness to label poisoning attacks.
Abstract (translated)
对抗恶意攻击的能力对于实际的分布式信号处理和机器学习系统至关重要。这类攻击的一个典型例子是标签投毒，即某些代理拥有被篡改的本地标签，并且会分享在其投毒数据上训练出来的模型。为了抵御恶意攻击，现有的研究工作往往集中在设计鲁棒聚合器；与此同时，加权平均聚合器通常被视为一种简单而脆弱的基础方法。 本文分析了在标签投毒攻击下的分布式梯度下降算法的稳健性，同时考虑了鲁棒和加权均值聚合器的效果。理论结果表明，鲁棒聚合器的学习误差依赖于网络拓扑结构，而加权均值聚合器的表现则与拓扑无关。值得注意的是，尽管加权平均聚合器通常被认为脆弱，但它在足够异质的情况下可以优于鲁棒聚合器，特别是在以下情况： (i) 全局投毒率（即整个网络中投毒代理的比例）低于局部投毒率（即常规代理的最大投毒邻居比例）； (ii) 常规代理的网络是断开连接的；或者 (iii) 常规代理的网络很稀疏且局部投毒率很高。 实验证据支持了我们的理论发现，突显了网络拓扑结构在抵御标签投毒攻击中的重要作用。
URL
https://arxiv.org/abs/2601.02682
PDF
-
Filtering Beats Fine Tuning: A Bayesian Kalman View of In Context Learning in LLMs
2026-01-02 21:18:48Andrew KirulutaAbstract
We present a theory-first framework that interprets inference-time adaptation in large language models (LLMs) as online Bayesian state estimation. Rather than modeling rapid adaptation as implicit optimization or meta-learning, we formulate task- and context-specific learning as the sequential inference of a low-dimensional latent adaptation state governed by a linearized state-space model. Under Gaussian assumptions, adaptation follows a Kalman recursion with closed-form updates for both the posterior mean and covariance. This perspective elevates epistemic uncertainty to an explicit dynamical variable. We show that inference-time learning is driven by covariance collapse, i.e., rapid contraction of posterior uncertainty induced by informative tokens, which typically precedes convergence of the posterior mean. Using observability conditions on token-level Jacobians, we establish stability of the Bayesian filter, prove exponential covariance contraction rates, and derive mean-square error bounds. Gradient descent, natural-gradient methods, and meta-learning updates arise as singular, noise-free limits of the filtering dynamics, positioning optimization-based adaptation as a degenerate approximation of Bayesian inference. The resulting theory provides a unified probabilistic account of in-context learning, parameter-efficient adaptation, and test-time learning without parameter updates. It yields explicit guarantees on stability and sample efficiency, offers a principled interpretation of prompt informativeness via information accumulation, and clarifies the role of uncertainty dynamics absent from existing accounts. Minimal illustrative experiments corroborate the qualitative predictions of the theory.
Abstract (translated)
我们提出了一种基于理论的框架，将大型语言模型（LLMs）在推理时的适应性解释为在线贝叶斯状态估计。不同于将快速适应视为隐式优化或元学习建模的方式，我们将特定任务和上下文的学习视为由线性化状态空间模型支配的低维潜在适应状态的顺序推断过程。在高斯假设下，适应遵循具有后验均值和协方差闭合形式更新的卡尔曼递归。这种视角将认识论不确定性提升为一个显式的动态变量。 我们表明，在推理时间学习是由协方差坍缩驱动的，即由信息令牌引起的后验不确定性的快速收缩，通常发生在后验均值收敛之前。通过在标记级别雅可比矩阵上的可观测性条件，我们建立了贝叶斯滤波器的稳定性，证明了指数级的协方差收缩率，并推导出了均方误差界限。 梯度下降、自然梯度方法以及元学习更新作为过滤动力学中无噪声极限的情况出现，将基于优化的方法适应视为贝叶斯推理的一种退化近似。由此产生的理论为上下文中的学习提供了一个统一的概率解释，涵盖了参数效率高且无需参数更新的测试时间学习，并提供了关于稳定性和样本效率的具体保证。 该理论通过信息积累的方式对提示信息性进行了原则性的诠释，并澄清了在现有论述中缺乏描述的不确定性动态的角色。最小的说明性实验验证了该理论的定性预测。
URL
https://arxiv.org/abs/2601.06100
PDF
-
Fast-weight Product Key Memory
2026-01-02 12:37:53Tianyu Zhao, Llion JonesAbstract
Sequence modeling layers in modern language models typically face a trade-off between storage capacity and computational efficiency. While Softmax attention offers unbounded storage at prohibitive quadratic costs, linear variants provide efficiency but suffer from limited, fixed-size storage. We propose Fast-weight Product Key Memory (FwPKM), a novel architecture that resolves this tension by transforming the sparse Product Key Memory (PKM) from a static module into a dynamic, "fast-weight" episodic memory. Unlike PKM, FwPKM updates its parameters dynamically at both training and inference time via local chunk-level gradient descent, allowing the model to rapidly memorize and retrieve new key-value pairs from input sequences. Experiments reveal that FwPKM functions as an effective episodic memory that complements the semantic memory of standard modules, yielding significant perplexity reductions on long-context datasets. Notably, in Needle in a Haystack evaluations, FwPKM generalizes to 128K-token contexts despite being trained on only 4K-token sequences.
Abstract (translated)
现代语言模型中的序列建模层通常在存储容量和计算效率之间存在权衡。虽然Softmax注意力提供了无界的存储能力，但其成本是高昂的二次复杂度；而线性变体虽提供高效性却受到有限且固定大小的存储限制。我们提出了一种新的架构——快速权重产品键记忆（FwPKM），它通过将稀疏的产品键记忆（PKM）从静态模块转化为动态的“快权重”情景记忆，解决了这一矛盾。不同于PKM，FwPKM在训练和推理过程中都能通过局部片段级别的梯度下降动态更新其参数，使模型能够快速存储并检索输入序列中的新键值对。 实验表明，FwPKM作为一种有效的情景记忆模块，在补充标准模块的语义记忆方面表现出色，并且在长上下文数据集上实现了显著的困惑度降低。值得注意的是，在Needle in a Haystack（大范围搜索）评估中，尽管仅基于4K令牌长度序列进行训练，FwPKM仍然能够泛化到128K令牌长度的情境中。
URL
https://arxiv.org/abs/2601.00671
PDF
-
Context Collapse: In-Context Learning and Model Collapse
Abstract
This thesis investigates two key phenomena in large language models (LLMs): in-context learning (ICL) and model collapse. We study ICL in a linear transformer with tied weights trained on linear regression tasks, and show that minimising the in-context loss leads to a phase transition in the learned parameters. Above a critical context length, the solution develops a skew-symmetric component. We prove this by reducing the forward pass of the linear transformer under weight tying to preconditioned gradient descent, and then analysing the optimal preconditioner. This preconditioner includes a skew-symmetric component, which induces a rotation of the gradient direction. For model collapse, we use martingale and random walk theory to analyse simplified settings - linear regression and Gaussian fitting - under both replacing and cumulative data regimes. We strengthen existing results by proving almost sure convergence, showing that collapse occurs unless the data grows sufficiently fast or is retained over time. Finally, we introduce the notion of context collapse: a degradation of context during long generations, especially in chain-of-thought reasoning. This concept links the dynamics of ICL with long-term stability challenges in generative models.
Abstract (translated)
这篇论文探讨了大型语言模型（LLM）中的两个关键现象：上下文学习（ICL）和模型崩溃。我们通过在训练线性回归任务的线性变压器中研究ICL，发现最小化上下文损失会导致学习参数发生相变。当上下文长度超过临界值时，解决方案会发展出一个斜对称分量。我们通过将权重绑定的线性变换器的前向传递简化为预条件梯度下降，并分析最优预条件器来证明这一点。该预条件器包含了一个斜对称分量，这会导致梯度方向发生旋转。 对于模型崩溃现象，我们利用鞅和随机游走理论在两种简化的设置下进行研究：线性回归和高斯拟合，在替换数据和累积数据的规则下。通过证明几乎确定收敛来加强现有结果，表明除非数据增长足够快或随时间保留，否则会发生崩溃。 最后，本文引入了上下文崩溃的概念：在长时间生成过程中，特别是链式思维推理中的上下文质量退化现象。这个概念将ICL的动力学与生成模型长期稳定性挑战联系起来。
URL
https://arxiv.org/abs/2601.00923
PDF
-
Nested Learning: The Illusion of Deep Learning Architectures
2025-12-31 07:59:43Ali Behrouz, Meisam Razaviyayn, Peilin Zhong, Vahab MirrokniAbstract
Despite the recent progresses, particularly in developing Language Models, there are fundamental challenges and unanswered questions about how such models can continually learn/memorize, self-improve, and find effective solutions. In this paper, we present a new learning paradigm, called Nested Learning (NL), that coherently represents a machine learning model with a set of nested, multi-level, and/or parallel optimization problems, each of which with its own context flow. Through the lenses of NL, existing deep learning methods learns from data through compressing their own context flow, and in-context learning naturally emerges in large models. NL suggests a philosophy to design more expressive learning algorithms with more levels, resulting in higher-order in-context learning and potentially unlocking effective continual learning capabilities. We advocate for NL by presenting three core contributions: (1) Expressive Optimizers: We show that known gradient-based optimizers, such as Adam, SGD with Momentum, etc., are in fact associative memory modules that aim to compress the gradients' information (by gradient descent). Building on this insight, we present other more expressive optimizers with deep memory and/or more powerful learning rules; (2) Self-Modifying Learning Module: Taking advantage of NL's insights on learning algorithms, we present a sequence model that learns how to modify itself by learning its own update algorithm; and (3) Continuum Memory System: We present a new formulation for memory system that generalizes the traditional viewpoint of long/short-term memory. Combining our self-modifying sequence model with the continuum memory system, we present a continual learning module, called Hope, showing promising results in language modeling, knowledge incorporation, and few-shot generalization tasks, continual learning, and long-context reasoning tasks.
Abstract (translated)
尽管在语言模型开发方面取得了近期的进展，但仍存在一些基本挑战和未解问题，这些问题涉及到如何让这些模型能够持续学习/记忆、自我改进以及找到有效的解决方案。在这篇论文中，我们提出了一种新的学习范式——嵌套学习（Nested Learning, NL），它通过一组嵌套的、多层次或多线程优化问题来一致地表示机器学习模型，每个问题都有其特定的上下文流。从NL的角度来看，现有的深度学习方法是通过对自身上下文流进行压缩来学习数据，而大模型中自然会出现上下文中的学习现象。NL提出了一种设计更具有表现力的学习算法的方法，这些算法可以有更多层次，从而实现更高阶的上下文学习，并有可能解锁有效的持续学习能力。 我们通过三个核心贡献来倡导嵌套学习（NL）： 1. 表达性优化器：我们知道基于梯度的优化器，如Adam、带有动量的SGD等，实际上都是关联记忆模块，旨在通过梯度下降压缩梯度信息。在此基础上，我们提出了其他更具有表现力的优化器，这些优化器包含深层记忆和/或更强的学习规则。 2. 自修改学习模块：利用NL对学习算法的理解，我们提出了一种序列模型，该模型能够通过学习自身的更新算法来自我修改。 3. 连续记忆系统：我们提出了一个新颖的记忆系统公式，它扩展了传统长短期记忆的观点。结合我们的自修改序列模型和连续记忆系统，我们提出了一种持续学习模块——希望（Hope），在语言建模、知识融合、少量示例泛化任务以及长时间上下文推理任务中表现出显著的性能。 这些贡献共同展示了嵌套学习范式如何推动机器学习技术向前发展，并提供了对未来研究方向的一些见解。
URL
https://arxiv.org/abs/2512.24695
PDF
-
HOLOGRAPH: Active Causal Discovery via Sheaf-Theoretic Alignment of Large Language Model Priors
2025-12-30 21:47:05Hyunjun KimAbstract
Causal discovery from observational data remains fundamentally limited by identifiability constraints. Recent work has explored leveraging Large Language Models (LLMs) as sources of prior causal knowledge, but existing approaches rely on heuristic integration that lacks theoretical grounding. We introduce HOLOGRAPH, a framework that formalizes LLM-guided causal discovery through sheaf theory--representing local causal beliefs as sections of a presheaf over variable subsets. Our key insight is that coherent global causal structure corresponds to the existence of a global section, while topological obstructions manifest as non-vanishing sheaf cohomology. We propose the Algebraic Latent Projection to handle hidden confounders and Natural Gradient Descent on the belief manifold for principled optimization. Experiments on synthetic and real-world benchmarks demonstrate that HOLOGRAPH provides rigorous mathematical foundations while achieving competitive performance on causal discovery tasks with 50-100 variables. Our sheaf-theoretic analysis reveals that while Identity, Transitivity, and Gluing axioms are satisfied to numerical precision (<10^{-6}), the Locality axiom fails for larger graphs, suggesting fundamental non-local coupling in latent variable projections. Code is available at [this https URL](this https URL).
Abstract (translated)
从观测数据中进行因果发现的根本限制在于识别性约束。最近的研究探索了利用大型语言模型（LLMs）作为先验因果知识来源的可能性，但现有的方法依赖于缺乏理论依据的启发式集成方式。我们提出了HOLOGRAPH框架，该框架通过层叠理论（sheaf theory）将基于LLM的因果发现形式化——表示局部因果信念为变量子集上的预层截面。我们的关键洞察是：一致的整体因果结构对应于全局截面的存在，而拓扑障碍则表现为非消失的层上同调。 我们提出代数潜在投影来处理隐藏混淆因素，并在信念流形上采用自然梯度下降进行原则性优化。在合成数据和真实世界基准上的实验表明，HOLOGRAPH提供了严格的数学基础，在涉及50到100个变量的因果发现任务中表现出竞争性的性能。 我们的层叠理论分析揭示了同一性、传递性和粘合公理以数值精度（<10^{-6}）被满足，而局部性公理在更大的图上失败，这表明潜在变量投影中的根本非局部耦合。代码可在[此链接](this https URL)获取。
URL
https://arxiv.org/abs/2512.24478
PDF
-
Trellis: Learning to Compress Key-Value Memory in Attention Models
2025-12-29 20:32:10Mahdi Karami, Ali Behrouz, Praneeth Kacham, Vahab MirrokniAbstract
Transformers, while powerful, suffer from quadratic computational complexity and the ever-growing Key-Value (KV) cache of the attention mechanism. This paper introduces Trellis, a novel Transformer architecture with bounded memory that learns how to compress its key-value memory dynamically at test time. Trellis replaces the standard KV cache with a fixed-size memory and train a two-pass recurrent compression mechanism to store new keys and values into memory. To achieve this, it leverages an online gradient descent procedure with a forget gate, enabling the compressed memory to be updated recursively while learning to retain important contextual information from incoming tokens at test time. Extensive experiments on language modeling, common-sense reasoning, recall-intensive tasks, and time series show that the proposed architecture outperforms strong baselines. Notably, its performance gains increase as the sequence length grows, highlighting its potential for long-context applications.
Abstract (translated)
尽管Transformer模型非常强大，但它们面临着二次计算复杂度和注意力机制中的键值(KV)缓存不断增长的问题。本文介绍了一种名为Trellis的新颖的Transformer架构，该架构具有有界内存，并且能够在测试时动态地学习如何压缩其键值记忆。Trellis用固定大小的记忆替换了标准KV缓存，并训练了一个两遍递归压缩机制来将新的键和值存储到记忆中。为了实现这一点，它利用了带有遗忘门的在线梯度下降过程，这使得在测试时能够递归地更新压缩后的内存并学习保留来自传入令牌的重要上下文信息。 通过在语言建模、常识推理、需要大量回忆的任务以及时间序列上的广泛实验表明，所提出的架构优于强大的基线模型。值得注意的是，其性能增益随着序列长度的增长而增加，这突显了它在处理长上下文应用方面的潜力。
URL
https://arxiv.org/abs/2512.23852
PDF
-
Towards Reliable Evaluation of Adversarial Robustness for Spiking Neural Networks
2025-12-27 08:43:06Jihang Wang, Dongcheng Zhao, Ruolin Chen, Qian Zhang, Yi ZengAbstract
Spiking Neural Networks (SNNs) utilize spike-based activations to mimic the brain's energy-efficient information processing. However, the binary and discontinuous nature of spike activations causes vanishing gradients, making adversarial robustness evaluation via gradient descent unreliable. While improved surrogate gradient methods have been proposed, their effectiveness under strong adversarial attacks remains unclear. We propose a more reliable framework for evaluating SNN adversarial robustness. We theoretically analyze the degree of gradient vanishing in surrogate gradients and introduce the Adaptive Sharpness Surrogate Gradient (ASSG), which adaptively evolves the shape of the surrogate function according to the input distribution during attack iterations, thereby enhancing gradient accuracy while mitigating gradient vanishing. In addition, we design an adversarial attack with adaptive step size under the $L_\infty$ constraint-Stable Adaptive Projected Gradient Descent (SA-PGD), achieving faster and more stable convergence under imprecise gradients. Extensive experiments show that our approach substantially increases attack success rates across diverse adversarial training schemes, SNN architectures and neuron models, providing a more generalized and reliable evaluation of SNN adversarial robustness. The experimental results further reveal that the robustness of current SNNs has been significantly overestimated and highlighting the need for more dependable adversarial training methods.
Abstract (translated)
脉冲神经网络（SNN）利用基于脉冲的激活来模仿大脑的能量高效信息处理方式。然而，脉冲激活的二进制和不连续特性导致梯度消失现象，使得通过梯度下降方法评估其对抗鲁棒性变得不可靠。尽管已经提出了改进的替代梯度方法，但在强对抗攻击下的有效性仍然不清楚。我们提出了一种更可靠的框架来评估SNN的对抗鲁棒性。理论上分析了替代梯度中的梯度消失程度，并引入自适应尖锐度替代梯度（ASSG），该方法根据输入分布动态调整替代函数形状，在攻击迭代过程中增强梯度精度同时减轻梯度消失问题。 此外，我们设计了一种具有自适应步长的$ L_\infty $约束下的对抗攻击——稳定自适应投影梯度下降法（SA-PGD），在不准确的梯度下实现了更快更稳定的收敛。大量实验表明，我们的方法显著提高了各种对抗训练方案、SNN架构和神经元模型中的攻击成功率，提供了对SNN对抗鲁棒性更为通用且可靠的评估方式。实验结果进一步揭示了当前SNN的稳健性被严重高估，并强调需要更加可靠地进行对抗训练的方法。
URL
https://arxiv.org/abs/2512.22522
PDF
-
Unifying Learning Dynamics and Generalization in Transformers Scaling Law
2025-12-26 17:20:09Chiwun YangAbstract
The scaling law, a cornerstone of Large Language Model (LLM) development, predicts improvements in model performance with increasing computational resources. Yet, while empirically validated, its theoretical underpinnings remain poorly understood. This work formalizes the learning dynamics of transformer-based language models as an ordinary differential equation (ODE) system, then approximates this process to kernel behaviors. Departing from prior toy-model analyses, we rigorously analyze stochastic gradient descent (SGD) training for multi-layer transformers on sequence-to-sequence data with arbitrary data distribution, closely mirroring real-world conditions. Our analysis characterizes the convergence of generalization error to the irreducible risk as computational resources scale with data, especially during the optimization process. We establish a theoretical upper bound on excess risk characterized by a distinct phase transition. In the initial optimization phase, the excess risk decays exponentially relative to the computational cost ${\sf C}$. However, once a specific resource allocation threshold is crossed, the system enters a statistical phase, where the generalization error follows a power-law decay of $\Theta(\mathsf{C}^{-1/6})$. Beyond this unified framework, our theory derives isolated scaling laws for model size, training time, and dataset size, elucidating how each variable independently governs the upper bounds of generalization.
Abstract (translated)
缩放法则，大型语言模型（LLM）开发的基石，预测了随着计算资源增加而改进模型性能的趋势。尽管这一理论已经在实证研究中得到了验证，但其背后的理论基础仍然不甚明了。这项工作将基于变压器的语言模型的学习动力学形式化为一个常微分方程（ODE）系统，并将其近似到核行为中去。不同于以往的玩具模型分析，我们严谨地分析了多层变压器在任意数据分布下的序列到序列数据上的随机梯度下降（SGD）训练过程，这更贴近现实世界的条件。 我们的研究描述了随着计算资源与数据量一同增加时泛化误差收敛至不可还原的风险的过程。并且，在优化过程中建立了由一个独特的相变特征定义的理论风险上限。在初始优化阶段，超额风险相对于计算成本${\sf C}$呈指数衰减形式。然而，一旦特定的资源配置阈值被跨越后，系统进入统计相位，在此阶段，泛化误差遵循$\Theta(\mathsf{C}^{-1/6})$的幂律衰退模式。 除了这个统一框架外，我们的理论还为模型大小、训练时间以及数据集规模分别推导出了孤立缩放法则，解释了每个变量如何独立地影响泛化的上限。
URL
https://arxiv.org/abs/2512.22088
PDF
-
MathLedger: A Verifiable Learning Substrate with Ledger-Attested Feedback
Abstract
Contemporary AI systems achieve extraordinary performance yet remain opaque and non-verifiable, creating a crisis of trust for safety-critical deployment. We introduce MathLedger, a substrate for verifiable machine cognition that integrates formal verification, cryptographic attestation, and learning dynamics into a single epistemic loop. The system implements Reflexive Formal Learning (RFL), a symbolic analogue of gradient descent where updates are driven by verifier outcomes rather than statistical loss. Phase I experiments validate the measurement and governance substrate under controlled conditions. CAL-EXP-3 validates measurement infrastructure (Delta p computation, variance tracking); separate stress tests confirm fail-closed governance triggers correctly under out-of-bounds conditions. No convergence or capability claims are made. The contribution is infrastructural: a working prototype of ledger-attested learning that enables auditability at scale. Keywords: verifiable learning, formal verification, cryptographic attestation, reflexive feedback, fail-closed governance
Abstract (translated)
当代人工智能系统实现了卓越的性能，但仍然缺乏透明性和可验证性，这在安全至关重要的部署中引发了信任危机。我们引入了MathLedger，这是一个可验证机器认知的基础架构，它将形式验证、加密证明和学习动力学整合到一个单一的认识循环中。该系统实施了反射式正式学习（RFL），这是一种符号类比的梯度下降方法，在这种方法中，更新是由验证结果驱动的，而不是统计损失。第一阶段实验在受控条件下验证了测量和治理基础架构的有效性。CAL-EXP-3 实验验证了测量基础设施（Delta p 计算、方差跟踪）；单独的压力测试确认了在超出范围条件下的关闭式治理触发机制正确运行。没有收敛或能力声明。这项贡献是基础设施性的：一个可审计的大型日志证明学习的工作原型。关键词：可验证学习，形式验证，加密证明，反射反馈，关闭式治理。
URL
https://arxiv.org/abs/2601.00816
PDF
-
Attention Is Not What You Need
2025-12-22 14:29:18Zhang ChongAbstract
We revisit a basic question in sequence modeling: is explicit self-attention actually necessary for strong performance and reasoning? We argue that standard multi-head attention is best seen as a form of tensor lifting: hidden vectors are mapped into a high-dimensional space of pairwise interactions, and learning proceeds by constraining this lifted tensor through gradient descent. This mechanism is extremely expressive but mathematically opaque, because after many layers it becomes very hard to describe the model with a small family of explicit invariants. To explore an alternative, we propose an attention-free architecture based on Grassmann flows. Instead of forming an L by L attention matrix, our Causal Grassmann layer (i) linearly reduces token states, (ii) encodes local token pairs as two-dimensional subspaces on a Grassmann manifold via Plucker coordinates, and (iii) fuses these geometric features back into the hidden states through gated mixing. Information therefore propagates by controlled deformations of low-rank subspaces over multi-scale local windows, so the core computation lives on a finite-dimensional manifold rather than in an unstructured tensor space. On the Wikitext-2 language modeling benchmark, purely Grassmann-based models with 13 to 18 million parameters achieve validation perplexities within about 10 to 15 percent of size-matched Transformers. On the SNLI natural language inference task, a Grassmann-Plucker head on top of DistilBERT slightly outperforms a Transformer head, with best validation and test accuracies of 0.8550 and 0.8538 compared to 0.8545 and 0.8511. We analyze the complexity of Grassmann mixing, show linear scaling in sequence length for fixed rank, and argue that such manifold-based designs offer a more structured route toward geometric and invariant-based interpretations of neural reasoning.
Abstract (translated)
我们重新审视了序列建模中的一个基本问题：显式的自注意力机制对于实现优异性能和推理而言是否真的必不可少？我们认为，标准的多头注意力机制最好被视为一种张量提升形式：隐藏向量被映射到一个高维空间，在这个空间中包含成对交互，并且通过梯度下降约束这种提升后的张量来进行学习。虽然这种机制非常具有表达能力，但由于在经过许多层后变得难以用一组显式的不变性来描述，因此从数学上讲它相当不透明。 为了探索一种替代方案，我们提出了一种基于Grassmann流的无注意力架构。与形成一个L×L的注意力矩阵不同，我们的因果Grassmann层（i）线性地减少令牌状态，（ii）通过Plücker坐标将局部令牌对编码为Grassmann流形上的二维子空间，并且（iii）通过门控混合将这些几何特征重新融合回隐藏状态。因此信息传播是通过对多尺度局部窗口中的低秩子空间进行受控变形来实现的，核心计算发生在有限维流形上而非无结构张量空间内。 在Wikitext-2语言建模基准测试中，仅基于Grassmann的模型（具有13至18百万参数）达到了约比同规模Transformer小10%到15%的有效困惑度。对于SNLI自然语言推理任务，在DistilBERT之上添加了一个Grassmann-Plücker头部，在验证和测试准确性上略微优于一个转换器头部，最佳验证与测试准确率分别为0.8550和0.8538，相比之下，后者为0.8545和0.8511。 我们分析了Grassmann混合的复杂性，并展示了对于固定秩，在序列长度上的线性缩放。我们认为基于流形的设计提供了一条更结构化的路径，能够朝着几何及不变性解释神经推理方向前进。
URL
https://arxiv.org/abs/2512.19428
PDF
-
Adversarial Robustness of Vision in Open Foundation Models
2025-12-19 18:59:16Jonathon Fox, William J Buchanan, Pavlos PapadopoulosAbstract
With the increase in deep learning, it becomes increasingly difficult to understand the model in which AI systems can identify objects. Thus, an adversary could aim to modify an image by adding unseen elements, which will confuse the AI in its recognition of an entity. This paper thus investigates the adversarial robustness of LLaVA-1.5-13B and Meta's Llama 3.2 Vision-8B-2. These are tested for untargeted PGD (Projected Gradient Descent) against the visual input modality, and empirically evaluated on the Visual Question Answering (VQA) v2 dataset subset. The results of these adversarial attacks are then quantified using the standard VQA accuracy metric. This evaluation is then compared with the accuracy degradation (accuracy drop) of LLaVA and Llama 3.2 Vision. A key finding is that Llama 3.2 Vision, despite a lower baseline accuracy in this setup, exhibited a smaller drop in performance under attack compared to LLaVA, particularly at higher perturbation levels. Overall, the findings confirm that the vision modality represents a viable attack vector for degrading the performance of contemporary open-weight VLMs, including Meta's Llama 3.2 Vision. Furthermore, they highlight that adversarial robustness does not necessarily correlate directly with standard benchmark performance and may be influenced by underlying architectural and training factors.
Abstract (translated)
随着深度学习的发展，理解AI系统如何识别对象的模型变得越来越困难。因此，对手可能会试图通过在图像中添加未见过的元素来修改图像，从而混淆AI对实体的识别能力。本文研究了LLaVA-1.5-13B和Meta的Llama 3.2 Vision-8B-2这两种AI系统在对抗攻击下的稳健性。它们针对视觉输入模式进行了无目标PGD（Projected Gradient Descent）测试，并在Visual Question Answering (VQA) v2数据集子集中进行经验评估。这些对抗攻击的结果使用标准的VQA准确率指标量化。这种评估结果与LLaVA和Llama 3.2 Vision的准确性下降（准确度降低）进行了比较。 一个关键发现是，尽管在该设置中的基线精度较低，但Llama 3.2 Vision在受到攻击时表现出更小的性能下降，特别是在更高的扰动水平下比LLaVA表现得更好。总体而言，这些研究结果证实了视觉模式可以作为破坏当代开放式权重VLM（Vision-Language Models）性能的有效攻击途径，包括Meta的Llama 3.2 Vision。此外，它们还强调了对抗稳健性不一定直接与标准基准性能相关联，并且可能受底层架构和训练因素的影响。
URL
https://arxiv.org/abs/2512.17902
PDF
-
Task Schema and Binding: A Double Dissociation Study of In-Context Learning
2025-12-19 08:14:21Chaeha KimAbstract
We provide causal mechanistic validation that in-context learning (ICL) decomposes into two separable mechanisms: Task Schema (abstract task type recognition) and Binding (specific input-output associations). Through activation patching experiments across 9 models from 7 Transformer families plus Mamba (370M-13B parameters), we establish three key findings: 1. Double dissociation: Task Schema transfers at 100% via late MLP patching; Binding transfers at 62% via residual stream patching -- proving separable mechanisms 2. Prior-Schema trade-off: Schema reliance inversely correlates with prior knowledge (Spearman rho = -0.596, p < 0.001, N=28 task-model pairs) 3. Architecture generality: The mechanism operates across all tested architectures including the non-Transformer Mamba These findings offer a mechanistic account of the ICL puzzle that contrasts with prior views treating ICL as a monolithic mechanism (whether retrieval-based, gradient descent-like, or purely Bayesian). By establishing that Schema and Binding are neurally dissociable -- not merely behavioral modes -- we provide causal evidence for dual-process theories of ICL. Models rely on Task Schema when prior knowledge is absent, but prior knowledge interferes through attentional mis-routing (72.7% recency bias) rather than direct output competition (0%). This explains why arbitrary mappings succeed (zero prior leads to full Schema reliance) while factual overrides fail -- and reveals that the true bottleneck is attentional, not output-level. Practical implications: Understanding these dual mechanisms enables more efficient prompt engineering -- reliable schema transfer reduces required demonstrations for novel tasks, while prior-aware design can mitigate the 38% binding failure rate in high-prior scenarios, improving ICL system reliability in production deployments.
Abstract (translated)
我们提供了关于基于上下文学习（ICL）的因果机制验证，证明ICL可以分解为两个可分离的机制：任务模式识别（Task Schema）和绑定（Binding特定输入-输出关联）。通过在来自7个Transformer家族以及Mamba模型系列中的9种不同架构（参数量从370M到13B不等）上进行激活补丁实验，我们得出了三个关键发现： 1. **双重分离**：任务模式识别可以通过对后期MLP层的补丁完全转移（100%），而绑定机制通过残差流补丁可以以62%的效率进行转移——这证明了它们是可分的机制。 2. **先验-模式权衡**：模式依赖性与先前知识存在反向相关关系（Spearman rho = -0.596，p < 0.001，N=28任务模型对），这意味着随着先前知识的增加，对于特定任务的任务模式识别的重要性会降低。 3. **架构通用性**：这种机制在所有测试过的架构中运行良好，包括非Transformer架构Mamba。 这些发现为ICL谜题提供了一个机械解释，并与之前将ICL视为单一机制（无论是基于检索、类似梯度下降还是纯粹贝叶斯）的观点相区别。通过证实模式和绑定从神经元层面上可以分离——不仅仅是行为模式的不同——我们为双过程理论提供了因果证据。 模型在缺乏先前知识时依赖于任务模式识别，但是当存在先前知识时会干扰注意力的正确路由（72.7%近期偏差），而不是直接输出竞争（0%）。这解释了为什么任意映射能够成功（零先验导致完全模式依赖）而事实覆盖失败，并揭示了真正的瓶颈在于注意层面而非输出层级。 **实用意义**：理解这些双机制可以更高效地进行提示工程——可靠的任务模式转移减少了对于新任务演示的需求，同时基于先前知识的设计可以在高先验场景中降低38%的绑定失败率，从而提高ICL系统在实际部署中的可靠性。
URL
https://arxiv.org/abs/2512.17325
PDF
-
ProbeMDE: Uncertainty-Guided Active Proprioception for Monocular Depth Estimation in Surgical Robotics
2025-12-12 18:36:53Britton Jordan, Jordan Thompson, Jesse F. d'Almeida, Hao Li, Nithesh Kumar, Susheela Sharma Stern, Ipek Oguz, Robert J. Webster III, Daniel Brown, Alan Kuntz, James FergusonAbstract
Monocular depth estimation (MDE) provides a useful tool for robotic perception, but its predictions are often uncertain and inaccurate in challenging environments such as surgical scenes where textureless surfaces, specular reflections, and occlusions are common. To address this, we propose ProbeMDE, a cost-aware active sensing framework that combines RGB images with sparse proprioceptive measurements for MDE. Our approach utilizes an ensemble of MDE models to predict dense depth maps conditioned on both RGB images and on a sparse set of known depth measurements obtained via proprioception, where the robot has touched the environment in a known configuration. We quantify predictive uncertainty via the ensemble's variance and measure the gradient of the uncertainty with respect to candidate measurement locations. To prevent mode collapse while selecting maximally informative locations to propriocept (touch), we leverage Stein Variational Gradient Descent (SVGD) over this gradient map. We validate our method in both simulated and physical experiments on central airway obstruction surgical phantoms. Our results demonstrate that our approach outperforms baseline methods across standard depth estimation metrics, achieving higher accuracy while minimizing the number of required proprioceptive measurements.
Abstract (translated)
单目深度估计（MDE）为机器人感知提供了一个有用的工具，但在手术场景等具有挑战性的环境中，其预测结果往往存在不确定性和不准确性。这些环境通常包含无纹理表面、镜面反射和遮挡等问题。为了应对这一问题，我们提出了ProbeMDE，这是一种成本意识型的主动感知框架，结合了RGB图像与稀疏本体感受测量数据来进行单目深度估计。 我们的方法利用一组MDE模型来预测密集的深度图，这些模型不仅依赖于RGB图像，还依赖于通过本体感受获取的一组稀疏已知深度测量值。机器人在特定配置下接触环境时可以获得这些测量值。我们通过集合方差量化预测不确定性，并计算候选测量位置处不确定性的梯度。为了防止选择最具有信息量的位置来进行本体感知（触觉）时出现模式崩溃，我们在该梯度图上使用Stein变分梯度下降法（SVGD）。 我们在模拟和物理实验中对中心气道阻塞手术模型进行了方法验证。我们的结果显示，相比基线方法，在标准深度估计指标方面，我们的方法在提高准确率的同时减少了所需的本体感知测量次数。
URL
https://arxiv.org/abs/2512.11773
PDF
-
Optimal Control and Structurally-Informed Gradient Optimization of a Custom 4-DOF Rigid-Body Manipulator
2025-12-12 03:15:02Brock Marcinczyk, Logan E. BeaverAbstract
This work develops a control-centric framework for a custom 4-DOF rigid-body manipulator by coupling a reduced-order Pontryagin's Maximum Principle (PMP) controller with a physics-informed Gradient Descent stage. The reduced PMP model provides a closed-form optimal control law for the joint accelerations, while the Gradient Descent module determines the corresponding time horizons by minimizing a cost functional built directly from the full Rigid-Body Dynamics. Structural-mechanics reaction analysis is used only to initialize feasible joint velocities-most critically the azimuthal component-ensuring that the optimizer begins in a physically admissible region. The resulting kinematic trajectories and dynamically consistent time horizons are then supplied to the symbolic Euler-Lagrange model to yield closed-form inverse-dynamics inputs. This pipeline preserves a strict control-theoretic structure while embedding the physical constraints and loading behavior of the manipulator in a computationally efficient way.
Abstract (translated)
这项工作开发了一个定制的四自由度刚体操作臂的控制中心框架，通过结合简化阶次的庞特里亚金最大值原理（PMP）控制器与物理信息梯度下降阶段来实现。简化的PMP模型提供了关节加速度的闭式最优控制律，而梯度下降模块则通过最小化一个直接从完整刚体动力学构建的成本函数来确定相应的时域范围。结构力学反应分析仅用于初始化可行的关节速度——尤其重要的是方位分量的速度，以确保优化器开始于物理上允许的区域内。随后生成的动力学一致的时间范围和运动学轨迹被提供给符号欧拉-拉格朗日模型，从而产生闭式反向动力学输入。此流程在保持严格的控制理论结构的同时，通过计算高效的方式嵌入操作臂的物理约束与加载行为。
URL
https://arxiv.org/abs/2512.11250
PDF
-
Arc Gradient Descent: A Mathematically Derived Reformulation of Gradient Descent with Phase-Aware, User-Controlled Step Dynamics
2025-12-07 09:03:45Nikhil Verma, Joonas Linnosmaa, Espinosa-Leal Leonardo, Napat VajraguptaAbstract
The paper presents the formulation, implementation, and evaluation of the ArcGD optimiser. The evaluation is conducted initially on a non-convex benchmark function and subsequently on a real-world ML dataset. The initial comparative study using the Adam optimiser is conducted on a stochastic variant of the highly non-convex and notoriously challenging Rosenbrock function, renowned for its narrow, curved valley, across dimensions ranging from 2D to 1000D and an extreme case of 50,000D. Two configurations were evaluated to eliminate learning-rate bias: (i) both using ArcGD's effective learning rate and (ii) both using Adam's default learning rate. ArcGD consistently outperformed Adam under the first setting and, although slower under the second, achieved super ior final solutions in most cases. In the second evaluation, ArcGD is evaluated against state-of-the-art optimizers (Adam, AdamW, Lion, SGD) on the CIFAR-10 image classification dataset across 8 diverse MLP architectures ranging from 1 to 5 hidden layers. ArcGD achieved the highest average test accuracy (50.7%) at 20,000 iterations, outperforming AdamW (46.6%), Adam (46.8%), SGD (49.6%), and Lion (43.4%), winning or tying on 6 of 8 architectures. Notably, while Adam and AdamW showed strong early convergence at 5,000 iterations, but regressed with extended training, whereas ArcGD continued improving, demonstrating generalization and resistance to overfitting without requiring early stopping tuning. Strong performance on geometric stress tests and standard deep-learning benchmarks indicates broad applicability, highlighting the need for further exploration. Moreover, it is also shown that a variant of ArcGD can be interpreted as a special case of the Lion optimiser, highlighting connections between the inherent mechanisms of such optimisation methods.
Abstract (translated)
这篇论文介绍了ArcGD优化器的公式化、实现和评估。评估首先在一个非凸基准函数上进行，随后在实际机器学习数据集上进行。最初的比较研究使用Adam优化器，在高度非凸且极具挑战性的Rosenbrock函数（以其狭窄而弯曲的山谷著称）的一个随机变体上展开，测试维度从2D到1000D不等，并包括极端情况下的50,000D。为了消除学习率偏差，评估了两种配置：(i) 两者均使用ArcGD的有效学习率；(ii) 两者均使用Adam的默认学习率。在第一种设置下，ArcGD始终优于Adam，在第二种设置下虽然速度较慢，但在大多数情况下仍达到了更优的最终解决方案。 在第二次评估中，ArcGD与最先进的优化器（包括Adam、AdamW、Lion和SGD）在CIFAR-10图像分类数据集上进行比较，涵盖了8种不同的多层感知机架构，从单隐藏层到五隐藏层。经过20,000次迭代后，ArcGD达到了最高的平均测试准确率（50.7%），超过了AdamW（46.6%）、Adam（46.8%）、SGD（49.6%）和Lion（43.4%），在八个架构中的六个中胜出或打平。值得注意的是，尽管Adam和AdamW在初始的5,000次迭代时表现出强烈的快速收敛性，在进一步训练后却退步了，而ArcGD则持续改进，显示出泛化能力和对过拟合的抗性，并且不需要提前停止调整。 几何压力测试和标准深度学习基准上的强劲表现表明其广泛适用性，突显了需要进行更多探索。此外，研究表明ArcGD的一个变体可以解释为Lion优化器的一种特殊情况，强调了此类优化方法内在机制之间的联系。
URL
https://arxiv.org/abs/2512.06737
PDF
-
CSMapping: Scalable Crowdsourced Semantic Mapping and Topology Inference for Autonomous Driving
2025-12-03 07:06:18Zhijian Qiao, Zehuan Yu, Tong Li, Chih-Chung Chou, Wenchao Ding, Shaojie ShenAbstract
Crowdsourcing enables scalable autonomous driving map construction, but low-cost sensor noise hinders quality from improving with data volume. We propose CSMapping, a system that produces accurate semantic maps and topological road centerlines whose quality consistently increases with more crowdsourced data. For semantic mapping, we train a latent diffusion model on HD maps (optionally conditioned on SD maps) to learn a generative prior of real-world map structure, without requiring paired crowdsourced/HD-map supervision. This prior is incorporated via constrained MAP optimization in latent space, ensuring robustness to severe noise and plausible completion in unobserved areas. Initialization uses a robust vectorized mapping module followed by diffusion inversion; optimization employs efficient Gaussian-basis reparameterization, projected gradient descent zobracket multi-start, and latent-space factor-graph for global consistency. For topological mapping, we apply confidence-weighted k-medoids clustering and kinematic refinement to trajectories, yielding smooth, human-like centerlines robust to trajectory variation. Experiments on nuScenes, Argoverse 2, and a large proprietary dataset achieve state-of-the-art semantic and topological mapping performance, with thorough ablation and scalability studies.
Abstract (translated)
群采（Crowdsourcing）能够实现自动驾驶地图的大规模构建，但低成本传感器噪声阻碍了随着数据量增加而提高的质量。为此，我们提出了CSMapping系统，该系统可以生成准确的语义地图和拓扑道路中心线，其质量会随着更多众包数据的加入而持续提升。 在语义地图方面，我们在高精度（HD）地图（可选条件为标准定义（SD）地图）上训练了一个潜在扩散模型，以学习现实世界中地图结构的生成先验，无需配对的众包/高清地图监督。该先验通过约束最大后验概率优化在潜在空间中进行整合，确保了对于严重噪声具有鲁棒性，并且能在未观察到的区域实现合理的完成。 初始化阶段采用了一种稳健的矢量化映射模块并结合扩散逆向操作；优化则采用了高效的高斯基重参数化、投影梯度下降和多起点策略以及潜在空间因子图来保证全局一致性。 在拓扑地图方面，我们通过应用基于置信度加权k-medoids聚类和运动学精炼的轨迹方法，生成了光滑且具有人类特征的道路中心线，并具备应对轨迹变化的能力。 实验结果表明，在nuScenes、Argoverse 2以及一个大型专有数据集上进行的测试中，CSMapping在语义地图和拓扑地图性能方面均达到了最先进的水平。研究还包含了详细的消融分析及规模性研究以证明其有效性。
URL
https://arxiv.org/abs/2512.03510
PDF
-
Rectifying LLM Thought from Lens of Optimization
2025-12-01 17:41:08Junnan Liu, Hongwei Liu, Songyang Zhang, Kai ChenAbstract
Recent advancements in large language models (LLMs) have been driven by their emergent reasoning capabilities, particularly through long chain-of-thought (CoT) prompting, which enables thorough exploration and deliberation. Despite these advances, long-CoT LLMs often exhibit suboptimal reasoning behaviors, such as overthinking and excessively protracted reasoning chains, which can impair performance. In this paper, we analyze reasoning processes through an optimization lens, framing CoT as a gradient descent procedure where each reasoning step constitutes an update toward problem resolution. Building on this perspective, we introduce RePro (Rectifying Process-level Reward), a novel approach to refine LLM reasoning during post-training. RePro defines a surrogate objective function to assess the optimization process underlying CoT, utilizing a dual scoring mechanism to quantify its intensity and stability. These scores are aggregated into a composite process-level reward, seamlessly integrated into reinforcement learning with verifiable rewards (RLVR) pipelines to optimize LLMs. Extensive experiments across multiple reinforcement learning algorithms and diverse LLMs, evaluated on benchmarks spanning mathematics, science, and coding, demonstrate that RePro consistently enhances reasoning performance and mitigates suboptimal reasoning behaviors.
Abstract (translated)
最近，大型语言模型（LLM）的进步主要得益于其通过长链条思考（CoT）提示所展现的新兴推理能力。尽管取得了这些进展，但使用长链式CoT的LLMs经常表现出次优的推理行为，例如过度思考和过于冗长的推理链，这可能会损害性能。在本文中，我们从优化的角度分析了推理过程，并将CoT视为一种梯度下降程序，在这种程序中，每个推理步骤都是向解决问题方向更新的一部分。在此基础上，我们引入了一种新的方法RePro（修正过程级奖励），该方法通过后训练来精炼LLM的推理能力。RePro定义了一个替代目标函数，用于评估CoT背后优化过程的质量，并使用一种双重评分机制来量化其强度和稳定性。这些分数被整合为一个综合的过程级奖励，并无缝集成到具有可验证回报（RLVR）管道中的强化学习过程中，以优化LLM。通过在多种强化学习算法及不同LLMs上进行的广泛实验，并且在数学、科学和编码等领域基准测试中进行了评估，我们发现RePro持续提高了推理性能并减轻了次优行为。
URL
https://arxiv.org/abs/2512.01925
PDF
