Gradient_Descent
-
Randomness and Interpolation Improve Gradient Descent
2025-10-14 23:32:01Jiawen Li, Pascal Lefevre, Anwar Pp Abdul MajeedAbstract
Based on Stochastic Gradient Descent (SGD), the paper introduces two optimizers, named Interpolational Accelerating Gradient Descent (IAGD) as well as Noise-Regularized Stochastic Gradient Descent (NRSGD). IAGD leverages second-order Newton Interpolation to expedite the convergence process during training, assuming relevancy in gradients between iterations. To avoid over-fitting, NRSGD incorporates a noise regularization technique that introduces controlled noise to the gradients during the optimization process. Comparative experiments of this research are conducted on the CIFAR-10, and CIFAR-100 datasets, benchmarking different CNNs(Convolutional Neural Networks) with IAGD and NRSGD against classical optimizers in Keras Package. Results demonstrate the potential of those two viable improvement methods in SGD, implicating the effectiveness of the advancements.
Abstract (translated)
基于随机梯度下降（SGD）算法，该论文提出了两种优化器：插值加速梯度下降法(IAGD)和噪声正则化随机梯度下降法(NRSGD)。IAGD利用了二阶牛顿插值技术来加快训练过程中的收敛速度，并假设不同迭代之间的梯度相关性。为了防止过拟合，NRSGD在优化过程中引入了一种噪声正则化技术，通过控制性的添加噪声到梯度中实现。 该研究的比较实验是在CIFAR-10和CIFAR-100数据集上进行的，并且使用了不同的卷积神经网络（CNNs）与Keras包中的经典优化器进行对比测试。结果表明，这两种改进SGD的方法具有潜在的价值，证明了这些方法的有效性及进步意义。
URL
https://arxiv.org/abs/2510.13040
PDF
-
Hardware-aware Coding Function Design for Compressive Single-Photon 3D Cameras
2025-10-14 03:52:24David Parra, Felipe Gutierrez-Barragan, Trevor Seets, Andreas VeltenAbstract
Single-photon cameras are becoming increasingly popular in time-of-flight 3D imaging because they can time-tag individual photons with extreme resolution. However, their performance is susceptible to hardware limitations, such as system bandwidth, maximum laser power, sensor data rates, and in-sensor memory and compute resources. Compressive histograms were recently introduced as a solution to the challenge of data rates through an online in-sensor compression of photon timestamp data. Although compressive histograms work within limited in-sensor memory and computational resources, they underperform when subjected to real-world illumination hardware constraints. To address this, we present a constrained optimization approach for designing practical coding functions for compressive single-photon 3D imaging. Using gradient descent, we jointly optimize an illumination and coding matrix (i.e., the coding functions) that adheres to hardware constraints. We show through extensive simulations that our coding functions consistently outperform traditional coding designs under both bandwidth and peak power constraints. This advantage is particularly pronounced in systems constrained by peak power. Finally, we show that our approach adapts to arbitrary parameterized impulse responses by evaluating it on a real-world system with a non-ideal impulse response function.
Abstract (translated)
单光子相机因能够以极高分辨率标记个别光子，在飞行时间3D成像领域越来越受欢迎。然而，它们的性能易受硬件限制的影响，如系统带宽、最大激光功率、传感器数据传输速率以及片上存储和计算资源等。最近引入了压缩直方图作为一种解决数据率挑战的方法，通过在线片上压缩光子时间戳数据来实现。尽管压缩直方图能够在有限的片上内存和计算资源下工作，但在面对实际照明硬件限制时表现不佳。为了解决这个问题，我们提出了一种约束优化方法，用于设计适用于压缩单光子3D成像的实际编码函数。通过梯度下降法，我们可以同时优化照明与编码矩阵（即编码功能），使其符合硬件限制条件。通过广泛的模拟实验，我们展示了我们的编码方案在带宽和峰值功率受限的情况下均优于传统编码设计方案的性能表现，特别是在受到峰值功率约束的系统中优势更为显著。最后，我们在一个具有非理想脉冲响应函数的实际系统上评估了这种方法，证明它可以适应任意参数化的脉冲响应。
URL
https://arxiv.org/abs/2510.12123
PDF
-
High-Dimensional Learning Dynamics of Quantized Models with Straight-Through Estimator
2025-10-12 16:43:46Yuma Ichikawa, Shuhei Kashiwamura, Ayaka SakataAbstract
Quantized neural network training optimizes a discrete, non-differentiable objective. The straight-through estimator (STE) enables backpropagation through surrogate gradients and is widely used. While previous studies have primarily focused on the properties of surrogate gradients and their convergence, the influence of quantization hyperparameters, such as bit width and quantization range, on learning dynamics remains largely unexplored. We theoretically show that in the high-dimensional limit, STE dynamics converge to a deterministic ordinary differential equation. This reveals that STE training exhibits a plateau followed by a sharp drop in generalization error, with plateau length depending on the quantization range. A fixed-point analysis quantifies the asymptotic deviation from the unquantized linear model. We also extend analytical techniques for stochastic gradient descent to nonlinear transformations of weights and inputs.
Abstract (translated)
量化神经网络训练优化的是一个离散且非可微的目标函数。直通估计器（STE）通过替代梯度使得反向传播成为可能，并被广泛使用。尽管先前的研究主要集中在替代梯度的性质及其收敛性上，但量化超参数（如位宽和量化范围）对学习动态的影响仍然很大程度上未被探索。我们从理论上证明，在高维极限下，STE动力学会收敛到一个确定性的常微分方程。这揭示了STE训练在泛化误差上会出现一个平台期后紧随其后的急剧下降的现象，并且这个平台期的长度取决于量化范围。通过固定点分析可以量化与未量化的线性模型的渐近偏离程度。我们还扩展了随机梯度下降的分析技术，以适用于权重和输入的非线性变换。
URL
https://arxiv.org/abs/2510.10693
PDF
-
Randomized HyperSteiner: A Stochastic Delaunay Triangulation Heuristic for the Hyperbolic Steiner Minimal Tree
2025-10-10 12:31:55Aniss Aiman Medbouhi, Alejandro Garc\'ia-Castellanos, Giovanni Luca Marchetti, Daniel Pelt, Erik J Bekkers, Danica KragicAbstract
We study the problem of constructing Steiner Minimal Trees (SMTs) in hyperbolic space. Exact SMT computation is NP-hard, and existing hyperbolic heuristics such as HyperSteiner are deterministic and often get trapped in locally suboptimal configurations. We introduce Randomized HyperSteiner (RHS), a stochastic Delaunay triangulation heuristic that incorporates randomness into the expansion process and refines candidate trees via Riemannian gradient descent optimization. Experiments on synthetic data sets and a real-world single-cell transcriptomic data show that RHS outperforms Minimum Spanning Tree (MST), Neighbour Joining, and vanilla HyperSteiner (HS). In near-boundary configurations, RHS can achieve a 32% reduction in total length over HS, demonstrating its effectiveness and robustness in diverse data regimes.
Abstract (translated)
我们研究了在双曲空间中构造斯特恩最小树（SMTs）的问题。精确的SMT计算是NP难问题，现有的如HyperSteiner之类的双曲启发式算法通常是确定性的，并且往往陷入局部次优解。我们引入了一种随机化HyPerSteiner (RHS) 方法，这是一种采用随机方法进行Delaunay三角剖分的启发式算法，在扩展过程中加入随机性并通过黎曼梯度下降优化来改进候选树。在合成数据集和实际单细胞转录组数据上的实验表明，与最小生成树（MST）、邻接连接法以及标准HyperSteiner (HS) 相比，RHS表现更优。在接近边界配置的情况下，RHS可以比HS减少32%的总长度，这证明了它在不同数据环境下的有效性和鲁棒性。
URL
https://arxiv.org/abs/2510.09328
PDF
-
Audible Networks: Deconstructing and Manipulating Sounds with Deep Non-Negative Autoencoders
2025-10-09 21:04:16Juan Jos\'e Burred, Carmine-Emanuele CellaAbstract
We propose the use of Non-Negative Autoencoders (NAEs) for sound deconstruction and user-guided manipulation of sounds for creative purposes. NAEs offer a versatile and scalable extension of traditional Non-Negative Matrix Factorization (NMF)-based approaches for interpretable audio decomposition. By enforcing non-negativity constraints through projected gradient descent, we obtain decompositions where internal weights and activations can be directly interpreted as spectral shapes and temporal envelopes, and where components can themselves be listened to as individual sound events. In particular, multi-layer Deep NAE architectures enable hierarchical representations with an adjustable level of granularity, allowing sounds to be deconstructed at multiple levels of abstraction: from high-level note envelopes down to fine-grained spectral details. This framework enables a wide new range of expressive, controllable, and randomized sound transformations. We introduce novel manipulation operations including cross-component and cross-layer synthesis, hierarchical deconstructions, and several randomization strategies that control timbre and event density. Through visualizations and resynthesis of practical examples, we demonstrate how NAEs can serve as flexible and interpretable tools for object-based sound editing.
Abstract (translated)
我们提出使用非负自动编码器（NAE）来进行声音分解，并允许用户引导式的音效操控，以满足创意需求。NAEs为传统的基于非负矩阵因式分解（NMF）的方法提供了一种灵活且可扩展的扩展，使得音频分解具有解释性。通过投影梯度下降方法实施非负约束，我们可以获得一种分解形式，在这种形式下内部权重和激活可以直接解释为频谱形状和时域包络，并且这些组件本身可以作为单独的声音事件被听取。特别是，多层深层NAE架构能够提供可调节颗粒度的分层表示，允许声音在多个抽象级别上进行拆解：从高层次音符包络到细微的频谱细节。这种框架使得一系列表达丰富、可控性强和随机化的声效转换成为可能。 我们引入了一些新的操作方法，包括跨组件和跨层次合成、分层分解以及几种控制音色和事件密度的随机化策略。通过实际案例的可视化与重合生成演示，我们展示了NAEs如何作为灵活且具有解释性的工具用于基于对象的声音编辑中。
URL
https://arxiv.org/abs/2510.08816
PDF
-
Active Next-Best-View Optimization for Risk-Averse Path Planning
2025-10-07 21:41:28Amirhossein Mollaei Khass, Guangyi Liu, Vivek Pandey, Wen Jiang, Boshu Lei, Kostas Daniilidis, Nader MoteeAbstract
Safe navigation in uncertain environments requires planning methods that integrate risk aversion with active perception. In this work, we present a unified framework that refines a coarse reference path by constructing tail-sensitive risk maps from Average Value-at-Risk statistics on an online-updated 3D Gaussian-splat Radiance Field. These maps enable the generation of locally safe and feasible trajectories. In parallel, we formulate Next-Best-View (NBV) selection as an optimization problem on the SE(3) pose manifold, where Riemannian gradient descent maximizes an expected information gain objective to reduce uncertainty most critical for imminent motion. Our approach advances the state-of-the-art by coupling risk-averse path refinement with NBV planning, while introducing scalable gradient decompositions that support efficient online updates in complex environments. We demonstrate the effectiveness of the proposed framework through extensive computational studies.
Abstract (translated)
在不确定环境中进行安全导航需要将风险规避与主动感知相结合的规划方法。本文提出了一种统一框架，通过在线更新的3D高斯光场（Gaussian-splat Radiance Field）上的平均值-at-风险统计量来构建尾部敏感的风险图，并用这些地图细化粗略参考路径，从而生成局部安全且可行的轨迹。同时，我们将下一个最佳视角（Next-Best-View, NBV）的选择表述为SE(3)姿态流形上的优化问题，在这个问题中使用黎曼梯度下降法来最大化预期信息增益的目标函数，以减少对即将发生的运动最为关键的不确定性。我们的方法通过将风险规避路径细化与NBV规划相结合，并引入可扩展的梯度分解来支持复杂环境中高效的在线更新，从而推进了当前的研究前沿。我们通过广泛的计算研究展示了所提框架的有效性。
URL
https://arxiv.org/abs/2510.06481
PDF
-
Improving Clinical Dataset Condensation with Mode Connectivity-based Trajectory Surrogates
2025-10-07 11:22:27Pafue Christy Nganjimi, Andrew Soltan, Danielle Belgrave, Lei Clifton, David A. Clifton, Anshul ThakurAbstract
Dataset condensation (DC) enables the creation of compact, privacy-preserving synthetic datasets that can match the utility of real patient records, supporting democratised access to highly regulated clinical data for developing downstream clinical models. State-of-the-art DC methods supervise synthetic data by aligning the training dynamics of models trained on real and those trained on synthetic data, typically using full stochastic gradient descent (SGD) trajectories as alignment targets; however, these trajectories are often noisy, high-curvature, and storage-intensive, leading to unstable gradients, slow convergence, and substantial memory overhead. We address these limitations by replacing full SGD trajectories with smooth, low-loss parametric surrogates, specifically quadratic Bézier curves that connect the initial and final model states from real training trajectories. These mode-connected paths provide noise-free, low-curvature supervision signals that stabilise gradients, accelerate convergence, and eliminate the need for dense trajectory storage. We theoretically justify Bézier-mode connections as effective surrogates for SGD paths and empirically show that the proposed method outperforms state-of-the-art condensation approaches across five clinical datasets, yielding condensed datasets that enable clinically effective model development.
Abstract (translated)
数据集浓缩（DC）能够创建紧凑且保护隐私的合成数据集，这些数据集可以与真实患者记录具有相同的效用，支持对高度监管的临床数据进行民主化访问，以开发下游临床模型。最先进的DC方法通过将基于真实数据和合成数据训练的模型的训练动态相匹配来监督生成的合成数据，通常使用完整的随机梯度下降（SGD）轨迹作为对齐目标；然而，这些轨迹往往噪声大、曲率高且占用大量存储空间，导致不稳定的梯度、缓慢收敛以及显著的记忆开销。我们通过用平滑且低损失的参数替代品替换完整的SGD轨迹来解决这些问题，具体来说是二次贝塞尔曲线，它连接基于真实训练轨迹初始和最终模型状态。这些模型相连路径提供无噪声、曲率低的监督信号，可以稳定梯度、加速收敛并消除对密集轨迹存储的需求。 我们从理论上证明了贝塞尔-模式连接作为SGD路径的有效替代品，并通过实验证明所提出的方法在五个临床数据集上超越了最先进的浓缩方法，在生成压缩的数据集方面表现出色，这些数据集能够支持有效的临床模型开发。
URL
https://arxiv.org/abs/2510.05805
PDF
-
From Noisy Traces to Stable Gradients: Bias-Variance Optimized Preference Optimization for Aligning Large Reasoning Models
2025-10-06 17:58:01Mingkang Zhu, Xi Chen, Bei Yu, Hengshuang Zhao, Jiaya JiaAbstract
Large reasoning models (LRMs) generate intermediate reasoning traces before producing final answers, yielding strong gains on multi-step and mathematical tasks. Yet aligning LRMs with human preferences, a crucial prerequisite for model deployment, remains underexplored. The statistically correct objective for preference alignment requires marginalizing over reasoning traces, but this computation is intractable in practice. A common workaround optimizes a single sampled trajectory, which introduces substantial gradient variance from stochastic trace sampling. To address this challenge, we frame preference optimization for LRMs through the lens of the bias--variance trade-off and propose Bias--Variance Optimized Preference Optimization (BVPO), a simple, drop-in method that mixes two gradient estimators: a high-variance trace-based estimator and a low-variance empty-trace estimator obtained by disabling reasoning trace generation. Our theory shows that BVPO strictly reduces trace-induced variance for any nontrivial mixture, provides a closed-form choice of the mixing weight that minimizes mean-squared error relative to the true marginal gradient, and under standard smoothness and step-size conditions, tightens classical convergence bounds for stochastic gradient descent. Empirically, BVPO improves alignment over the best baseline by up to 7.8 points on AlpacaEval~2 and 6.8 points on Arena-Hard. Despite being trained only on general conversational data, BVPO also boosts reasoning performance for base models by up to 4.0 points on the average of six math reasoning benchmarks. These results identify variance from trace sampling as a key bottleneck and demonstrate that directly optimizing the bias--variance trade-off yields more stable training and stronger overall performance.
Abstract (translated)
大型推理模型（LRMs）在生成最终答案之前会先生成中间的推理痕迹，这在多步骤和数学任务中表现出色。然而，将这些模型与人类偏好对齐仍是一个未被充分研究的问题，这是部署模型的一个关键前提条件。统计上正确的对齐目标需要对推理痕迹进行边缘化处理，但在实践中这种计算是不可行的。常见的替代方案是优化单一抽样轨迹，但这引入了从随机痕迹采样中产生的大量梯度方差。 为了解决这个问题，我们通过偏差-方差权衡的角度来审视LRMs中的偏好优化，并提出了偏差-方差优化的偏好优化（BVPO），这是一种简单的方法，可以在不改变原有模型结构的情况下使用。该方法结合了两种梯度估计器：高方差痕迹基估计器和低方差空痕估计器（通过禁用推理痕迹生成获得）。我们的理论表明，对于任何非平凡混合物，BVPO可以严格降低由痕迹引起的方差，并提供了一个闭式的权重选择形式，以最小化相对于真实边际梯度的均方误差。在标准平滑性和步长条件下，它还可以紧缩经典随机梯度下降的收敛界限。 实证研究显示，在AlpacaEval~2和Arena-Hard任务中，BVPO比最佳基线提高了最多7.8分和6.8分。即使只用一般的对话数据进行训练，BVPO也能够提升基础模型在六个数学推理基准测试平均分数上的表现，最多提升了4.0分。 这些结果表明，从痕迹采样中产生的方差是主要的瓶颈问题，并证明了直接优化偏差-方差权衡可以带来更稳定的训练和更强的整体性能。
URL
https://arxiv.org/abs/2510.05095
PDF
-
Adaptively Sampling-Reusing-Mixing Decomposed Gradients to Speed Up Sharpness Aware Minimization
2025-10-04 10:08:14Jiaxin Deng, Junbiao PangAbstract
Sharpness-Aware Minimization (SAM) improves model generalization but doubles the computational cost of Stochastic Gradient Descent (SGD) by requiring twice the gradient calculations per optimization step. To mitigate this, we propose Adaptively sampling-Reusing-mixing decomposed gradients to significantly accelerate SAM (ARSAM). Concretely, we firstly discover that SAM's gradient can be decomposed into the SGD gradient and the Projection of the Second-order gradient onto the First-order gradient (PSF). Furthermore, we observe that the SGD gradient and PSF dynamically evolve during training, emphasizing the growing role of the PSF to achieve a flat minima. Therefore, ARSAM is proposed to the reused PSF and the timely updated PSF still maintain the model's generalization ability. Extensive experiments show that ARSAM achieves state-of-the-art accuracies comparable to SAM across diverse network architectures. On CIFAR-10/100, ARSAM is comparable to SAM while providing a speedup of about 40\%. Moreover, ARSAM accelerates optimization for the various challenge tasks (\textit{e.g.}, human pose estimation, and model quantization) without sacrificing performance, demonstrating its broad practicality.% The code is publicly accessible at: this https URL.
Abstract (translated)
Sharpness-Aware Minimization (SAM) enhances model generalization but doubles the computational cost of Stochastic Gradient Descent (SGD) by necessitating twice as many gradient calculations per optimization step. To address this issue, we propose Adaptively Sampling-Reusing-Mixing Decomposed Gradients to significantly accelerate SAM (ARSAM). Specifically, we first discover that SAM's gradients can be decomposed into the SGD gradient and the Projection of the Second-order Gradient onto the First-order Gradient (PSF). Additionally, we observe that both the SGD gradient and PSF dynamically change during training, with the role of PSF becoming increasingly significant to achieve a flatter minimum. ARSAM is designed to effectively reuse old PSFs while ensuring that newly updated PSFs maintain the model's generalization capabilities. Extensive experiments demonstrate that ARSAM achieves state-of-the-art accuracy comparable to SAM across various network architectures. On datasets such as CIFAR-10 and CIFAR-100, ARSAM matches SAM in performance but offers a 40% speedup. Furthermore, ARSAM accelerates optimization for diverse challenging tasks (e.g., human pose estimation and model quantization) without compromising performance, showcasing its broad practical applicability. (Note: The original text mentions that the code is publicly accessible at a specific URL. If you need the link to be included or have another way of accessing it, please provide additional details.)
URL
https://arxiv.org/abs/2510.03763
PDF
-
Dale meets Langevin: A Multiplicative Denoising Diffusion Model
2025-10-03 05:23:33Nishanth Shetty, Madhava Prasath, Chandra Sekhar SeelamantulaAbstract
Gradient descent has proven to be a powerful and effective technique for optimization in numerous machine learning applications. Recent advances in computational neuroscience have shown that learning in standard gradient descent optimization formulation is not consistent with learning in biological systems. This has opened up interesting avenues for building biologically inspired learning techniques. One such approach is inspired by Dale's law, which states that inhibitory and excitatory synapses do not swap roles during the course of learning. The resulting exponential gradient descent optimization scheme leads to log-normally distributed synaptic weights. Interestingly, the density that satisfies the Fokker-Planck equation corresponding to the stochastic differential equation (SDE) with geometric Brownian motion (GBM) is the log-normal density. Leveraging this connection, we start with the SDE governing geometric Brownian motion, and show that discretizing the corresponding reverse-time SDE yields a multiplicative update rule, which surprisingly, coincides with the sampling equivalent of the exponential gradient descent update founded on Dale's law. Furthermore, we propose a new formalism for multiplicative denoising score-matching, subsuming the loss function proposed by Hyvaerinen for non-negative data. Indeed, log-normally distributed data is positive and the proposed score-matching formalism turns out to be a natural fit. This allows for training of score-based models for image data and results in a novel multiplicative update scheme for sample generation starting from a log-normal density. Experimental results on MNIST, Fashion MNIST, and Kuzushiji datasets demonstrate generative capability of the new scheme. To the best of our knowledge, this is the first instance of a biologically inspired generative model employing multiplicative updates, founded on geometric Brownian motion.
Abstract (translated)
梯度下降法已经被证明是众多机器学习应用中一种强大且有效的优化技术。然而，近年来计算神经科学的进展表明，在标准的梯度下降优化框架下所实现的学习与生物系统中的学习并不一致。这为构建受生物学启发的学习方法开辟了新的途径。其中一种方法受到达勒定律（Dale's law）的启发，该定律指出在学习过程中抑制性突触和兴奋性突触不会互换角色。由此产生的指数梯度下降优化方案导致突触权重呈对数正态分布。值得注意的是，与几何布朗运动（GBM）随机微分方程对应的Fokker-Planck方程的密度是对数正态密度。 利用这一联系，我们从描述几何布朗运动的随机微分方程开始，然后证明了相应反向时间随机微分方程离散化后得到一个乘法更新规则。令人惊讶的是，这个更新规则与基于达勒定律的指数梯度下降采样等效性一致。此外，我们还提出了一种新的乘法去噪评分匹配形式，它包含了Hyvaerinen提出的用于非负数据的损失函数。事实上，对数正态分布的数据为正值，并且所提的评分匹配方法正好适用。这使得基于得分模型训练图像数据成为可能，并产生了一个新颖的从对数正态密度开始进行样本生成的乘法更新方案。 在MNIST、Fashion MNIST和Kuzushiji等数据集上的实验结果表明，新方案具有良好的生成能力。据我们所知，这是首次将受生物启发的生成模型与基于几何布朗运动的乘法更新方法相结合的应用实例。
URL
https://arxiv.org/abs/2510.02730
PDF
-
To Compress or Not? Pushing the Frontier of Lossless GenAI Model Weights Compression with Exponent Concentration
2025-10-03 02:22:13Zeyu Yang, Tianyi Zhang, Jianwen Xie, Chuan Li, Zhaozhuo Xu, Anshumali ShrivastavaAbstract
The scaling of Generative AI (GenAI) models into the hundreds of billions of parameters makes low-precision computation indispensable for efficient deployment. We argue that the fundamental solution lies in developing low-precision floating-point formats, which inherently provide numerical stability, memory savings, and hardware efficiency without dequantization overhead. In this paper, we present a theoretical and empirical study of an exponent concentration phenomenon in GenAI weights: exponents consistently exhibit low entropy across architectures and modalities. We show that this arises naturally from $\alpha$-stable distributions induced by stochastic gradient descent, and we prove tight bounds on the entropy of exponents. Our analysis establishes a theoretical compression limit near FP4.67, which motivates the design of a practical FP8 format. Building on these insights, we propose Exponent-Concentrated FP8 (ECF8), a lossless compression framework with entropy-aware encoding and GPU-optimized decoding. Experiments on LLMs and DiTs up to 671B parameters demonstrate up to 26.9% memory savings and 177.1% throughput acceleration, with perfectly lossless computations, i.e., no deviation in model outputs. Our results establish exponent concentration as a statistical law of trained models and open a principled path for lossless low-precision floating-point design in the FP8 era.
Abstract (translated)
生成式AI（GenAI）模型的参数量扩展到数百亿级别，使得低精度计算成为高效部署不可或缺的一部分。我们认为，根本性的解决方案在于开发提供数值稳定性、内存节省和硬件效率的低精度浮点格式，并且无需去量化开销。在本文中，我们提出了关于GenAI权重中的指数集中现象的一个理论与实证研究：无论架构还是模式如何，指数部分始终表现出较低的熵值。我们展示了这种现象自然地由随机梯度下降所诱导的α-稳定分布产生，并证明了指数熵值得紧致界。我们的分析建立了接近FP4.67的理论压缩极限，从而激励设计一个实用的FP8格式。基于这些见解，我们提出了指数集中型FP8（ECF8），这是一个无损压缩框架，具备熵感知编码和GPU优化解码功能。在高达671B参数量的LLM和DiT上的实验表明，最多可节省26.9%的内存，并且性能加速可达177.1%，同时确保完全无损计算，即模型输出没有偏差变化。我们的结果确立了指数集中作为训练模型中的统计规律的地位，并为FP8时代的无损低精度浮点设计开辟了一条有原则的道路。
URL
https://arxiv.org/abs/2510.02676
PDF
-
Equilibrium Matching: Generative Modeling with Implicit Energy-Based Models
2025-10-02 17:59:06Runqian Wang, Yilun DuAbstract
We introduce Equilibrium Matching (EqM), a generative modeling framework built from an equilibrium dynamics perspective. EqM discards the non-equilibrium, time-conditional dynamics in traditional diffusion and flow-based generative models and instead learns the equilibrium gradient of an implicit energy landscape. Through this approach, we can adopt an optimization-based sampling process at inference time, where samples are obtained by gradient descent on the learned landscape with adjustable step sizes, adaptive optimizers, and adaptive compute. EqM surpasses the generation performance of diffusion/flow models empirically, achieving an FID of 1.90 on ImageNet 256$\times$256. EqM is also theoretically justified to learn and sample from the data manifold. Beyond generation, EqM is a flexible framework that naturally handles tasks including partially noised image denoising, OOD detection, and image composition. By replacing time-conditional velocities with a unified equilibrium landscape, EqM offers a tighter bridge between flow and energy-based models and a simple route to optimization-driven inference.
Abstract (translated)
我们介绍了一种新的生成模型框架——平衡匹配（EqM），该框架从平衡动力学的角度构建。与传统的扩散和基于流的生成模型不同，EqM摒弃了非平衡的时间条件动力学，并学习隐式能量景观的平衡梯度。通过这种方法，在推理过程中可以采用基于优化的采样过程，即通过对学习到的能量景观进行可调整步长、自适应优化器及计算量的梯度下降来获取样本。 实验结果表明，EqM在生成性能上超过了扩散和流模型，在ImageNet 256×256数据集上的FID（Frechet Inception Distance）得分为1.90。从理论上讲，EqM被证明可以从数据流形中学习并进行采样。除生成任务外，EqM还是一个灵活的框架，能够自然地处理部分噪声图像去噪、异常检测和图像合成等任务。 通过用统一的能量景观取代时间条件速度，EqM建立了流模型与能量模型之间更紧密的联系，并提供了一条简单且基于优化推断的方法路径。
URL
https://arxiv.org/abs/2510.02300
PDF
-
CurES: From Gradient Analysis to Efficient Curriculum Learning for Reasoning LLMs
2025-10-01 15:41:27Yongcheng Zeng, Zexu Sun, Bokai Ji, Erxue Min, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Haifeng Zhang, Xu Chen, Jun WangAbstract
Curriculum learning plays a crucial role in enhancing the training efficiency of large language models (LLMs) on reasoning tasks. However, existing methods often fail to adequately account for variations in prompt difficulty or rely on simplistic filtering mechanisms to select prompt datasets within a narrow criterion range, resulting in significant computational waste. In this work, we approach the problem from the perspective of reinforcement learning gradient optimization, offering a systematic and theoretical investigation into how to improve the training efficiency of LLMs. We identify two key factors influencing training efficiency: the selection of training prompts and the allocation of rollout quantities across different prompts. Our theoretical analysis reveals that the sampling distribution of prompts dictates the convergence rate of gradient descent, while the allocation of the rollout quantity influences the consistency and stability of overall gradient updates. Based on these insights, we propose CurES, an efficient training method that accelerates convergence and employs Bayesian posterior estimation to minimize computational overhead. Experiments demonstrate that our CurES outperforms Group Relative Policy Optimization (GRPO) by \textbf{+3.30} points and \textbf{+4.82} points with 1.5B and 7B models, respectively. Additionally, CurES exhibits faster convergence compared to baselines, including GRPO.
Abstract (translated)
课程学习在提高大规模语言模型（LLM）在推理任务上的训练效率方面起着关键作用。然而，现有的方法常常无法充分考虑到提示难度的差异，或者依赖于简单的过滤机制来选择特定标准范围内的提示数据集，这导致了显著的计算浪费。在这项工作中，我们从强化学习梯度优化的角度出发，对如何提高LLM的训练效率进行了系统的理论研究。我们确定了影响训练效率的两个关键因素：训练提示的选择和不同提示下rollout数量的分配。我们的理论分析揭示了提示采样分布决定了梯度下降的收敛速度，而rollout数量的分配则影响整体梯度更新的一致性和稳定性。 基于这些见解，我们提出了CurES这一高效的训练方法，它通过加速收敛并采用贝叶斯后验估计来最小化计算开销。实验结果表明，在1.5B和7B模型上，我们的CurES方法分别比Group Relative Policy Optimization (GRPO) 方法高出3.30分和4.82分。此外，与包括GRPO在内的基线相比，CurES展示了更快的收敛速度。
URL
https://arxiv.org/abs/2510.01037
PDF
-
Directed-MAML: Meta Reinforcement Learning Algorithm with Task-directed Approximation
2025-09-30 19:42:15Yang Zhang, Huiwen Yan, Mushuang LiuAbstract
Model-Agnostic Meta-Learning (MAML) is a versatile meta-learning framework applicable to both supervised learning and reinforcement learning (RL). However, applying MAML to meta-reinforcement learning (meta-RL) presents notable challenges. First, MAML relies on second-order gradient computations, leading to significant computational and memory overhead. Second, the nested structure of optimization increases the problem's complexity, making convergence to a global optimum more challenging. To overcome these limitations, we propose Directed-MAML, a novel task-directed meta-RL algorithm. Before the second-order gradient step, Directed-MAML applies an additional first-order task-directed approximation to estimate the effect of second-order gradients, thereby accelerating convergence to the optimum and reducing computational cost. Experimental results demonstrate that Directed-MAML surpasses MAML-based baselines in computational efficiency and convergence speed in the scenarios of CartPole-v1, LunarLander-v2 and two-vehicle intersection crossing. Furthermore, we show that task-directed approximation can be effectively integrated into other meta-learning algorithms, such as First-Order Model-Agnostic Meta-Learning (FOMAML) and Meta Stochastic Gradient Descent(Meta-SGD), yielding improved computational efficiency and convergence speed.
Abstract (translated)
模型无关元学习（Model-Agnostic Meta-Learning，MAML）是一个适用于监督学习和强化学习（RL）的灵活元学习框架。然而，在将MAML应用于元强化学习（meta-RL）时会遇到显著挑战。首先，MAML依赖于二阶导数计算，这会导致较大的计算量和内存开销。其次，优化过程中的嵌套结构增加了问题复杂性，使得达到全局最优解更加困难。 为了克服这些限制，我们提出了Directed-MAML，一种新颖的任务导向型元强化学习算法。在进行二阶梯度步骤之前，Directed-MAML应用一个额外的一阶任务导向近似来估计二阶导数的影响，从而加速收敛到最佳点并减少计算成本。实验结果表明，在CartPole-v1、LunarLander-v2及两车交叉路口等场景中，Directed-MAML在计算效率和收敛速度方面均超越了基于MAML的基准算法。 此外，我们还展示了任务导向近似可以有效地集成到其他元学习算法中，例如一阶模型无关元学习（First-Order Model-Agnostic Meta-Learning，FOMAML）和元随机梯度下降（Meta Stochastic Gradient Descent，Meta-SGD），从而提高计算效率并加快收敛速度。
URL
https://arxiv.org/abs/2510.00212
PDF
-
Ascent Fails to Forget
2025-09-30 15:48:49Ioannis Mavrothalassitis, Pol Puigdemont, Noam Itzhak Levi, Volkan CevherAbstract
Contrary to common belief, we show that gradient ascent-based unconstrained optimization methods frequently fail to perform machine unlearning, a phenomenon we attribute to the inherent statistical dependence between the forget and retain data sets. This dependence, which can manifest itself even as simple correlations, undermines the misconception that these sets can be independently manipulated during unlearning. We provide empirical and theoretical evidence showing these methods often fail precisely due to this overlooked relationship. For random forget sets, this dependence means that degrading forget set metrics (which, for a retrained model, should mirror test set metrics) inevitably harms overall test performance. Going beyond random sets, we consider logistic regression as an instructive example where a critical failure mode emerges: inter-set dependence causes gradient descent-ascent iterations to progressively diverge from the ideal retrained model. Strikingly, these methods can converge to solutions that are not only far from the retrained ideal but are potentially even further from it than the original model itself, rendering the unlearning process actively detrimental. A toy example further illustrates how this dependence can trap models in inferior local minima, inescapable via finetuning. Our findings highlight that the presence of such statistical dependencies, even when manifest only as correlations, can be sufficient for ascent-based unlearning to fail. Our theoretical insights are corroborated by experiments on complex neural networks, demonstrating that these methods do not perform as expected in practice due to this unaddressed statistical interplay.
Abstract (translated)
与普遍看法相反，我们展示了基于梯度上升的无约束优化方法在执行机器遗忘（machine unlearning）时经常失败的现象。我们将这一现象归因于忘记集和保留集之间的固有统计依赖性。这种依赖关系即使表现为简单的相关性，也会破坏这样一种误解，即这些集合可以在不学习过程中独立操作。我们提供了实证和理论证据，表明正是这种被忽视的关系导致了方法的失效。 对于随机选择的遗忘集而言，这种依赖意味着降低忘记集指标（在重新训练模型时应与测试集指标一致）不可避免地会损害整体测试性能。超越随机集合的情况，我们将逻辑回归作为示例来说明一个重要失败模式：跨集依赖性导致梯度下降-上升迭代逐渐偏离理想重新训练后的模型。值得注意的是，这些方法可以收敛到距离重新训练的理想结果很远甚至比原始模型还差的解决方案上，使不学习过程变得有害。 一个玩具例子进一步展示了这种依赖如何将模型困在次优局部最小值中，无法通过微调逃脱。我们的研究发现强调了即使只表现为相关性，这样的统计依赖关系就足以导致基于上升的方法失效。我们理论见解得到了在复杂神经网络上的实验验证，这些实验表明由于未解决的统计交互作用，在实践中这些方法的表现并不如预期。 简而言之，研究表明传统优化方法在执行机器遗忘任务时常常失败，并且这种失败是由数据集间的统计依赖性引起的，这与人们通常认为的数据集可以独立处理的看法相悖。
URL
https://arxiv.org/abs/2509.26427
PDF
-
Stable Forgetting: Bounded Parameter-Efficient Unlearning in LLMs
2025-09-29 01:30:15Arpit Garg, Hemanth Saratchandran, Ravi Garg, Simon LuceyAbstract
Machine unlearning in large language models (LLMs) is essential for privacy and safety; however, existing approaches remain unstable and unreliable. A widely used strategy, the gradient difference method, applies gradient descent on retained data while performing gradient ascent on forget data, the data whose influence should be removed. However, when combined with cross-entropy loss, this procedure causes unbounded growth of weights and gradients, leading to training instability and degrading both forgetting and retention. We provide a theoretical framework that explains this failure, explicitly showing how ascent on the forget set destabilizes optimization in the feedforward MLP layers of LLMs. Guided by this insight, we propose Bounded Parameter-Efficient Unlearning, a parameter-efficient approach that stabilizes LoRA-based fine-tuning by applying bounded functions to MLP adapters. This simple modification controls the weight dynamics during ascent, enabling the gradient difference method to converge reliably. Across the TOFU, TDEC, and MUSE benchmarks, and across architectures and scales from 125M to 8B parameters, our method achieves substantial improvements in forgetting while preserving retention, establishing a novel theoretically grounded and practically scalable framework for unlearning in LLMs.
Abstract (translated)
大型语言模型（LLMs）中的机器遗忘对于隐私和安全至关重要；然而，现有的方法仍然不稳定且不可靠。广泛使用的一种策略是梯度差分法，在保留的数据上应用梯度下降，同时在应被删除影响的“忘记”数据上执行梯度上升。但是，当与交叉熵损失结合时，这种方法会导致权重和梯度无界增长，从而引起训练不稳定，并降低遗忘和保持的效果。我们提供了一个理论框架来解释这一失败，明确展示了对“忘记集”的上升如何在前馈MLP层中破坏优化的稳定性。 基于这一见解，我们提出了有界的参数高效遗忘（Bounded Parameter-Efficient Unlearning）方法，这是一种参数高效的策略，通过将有界函数应用于LoRA微调中的MLP适配器来稳定基于LoRA的微调。这一简单的修改控制了上升过程中的权重动态，使得梯度差分法能够可靠地收敛。 在TOFU、TDEC和MUSE基准测试中，以及在从125M到80亿参数的不同架构和规模上，我们的方法实现了显著的遗忘改进，并保持了保留的效果。这建立了一个理论基础坚实且实践可扩展的新框架，用于LLMs中的机器遗忘。
URL
https://arxiv.org/abs/2509.24166
PDF
-
Gradient Flow Convergence Guarantee for General Neural Network Architectures
2025-09-28 13:52:13Yash JakhmolaAbstract
A key challenge in modern deep learning theory is to explain the remarkable success of gradient-based optimization methods when training large-scale, complex deep neural networks. Though linear convergence of such methods has been proved for a handful of specific architectures, a united theory still evades researchers. This article presents a unified proof for linear convergence of continuous gradient descent, also called gradient flow, while training any neural network with piecewise non-zero polynomial activations or ReLU, sigmoid activations. Our primary contribution is a single, general theorem that not only covers architectures for which this result was previously unknown but also consolidates existing results under weaker assumptions. While our focus is theoretical and our results are only exact in the infinitesimal step size limit, we nevertheless find excellent empirical agreement between the predictions of our result and those of the practical step-size gradient descent method.
Abstract (translated)
现代深度学习理论面临的一个关键挑战是解释基于梯度的优化方法在训练大规模复杂深层神经网络时取得的成功。尽管已证明了针对少数特定架构的线性收敛，但一个统一的理论仍然困扰着研究人员。本文提出了一种对连续梯度下降（又称梯度流）进行线性收敛统一证明的方法，在使用分段非零多项式激活函数或ReLU、Sigmoid激活函数训练任何神经网络时有效。我们的主要贡献是一个单一的一般定理，该定理不仅涵盖了此前未知结果的架构类型，还以更弱的前提假设统一了现有的研究成果。尽管我们侧重于理论研究，并且我们的结论仅在微小步长极限下才精确成立，但我们发现从这种方法得出的结果与实际步长梯度下降方法得到的经验结果有极好的一致性。
URL
https://arxiv.org/abs/2509.23887
PDF
-
Adaptive Token-Weighted Differential Privacy for LLMs: Not All Tokens Require Equal Protection
2025-09-27 10:51:07Manjiang Yu, Priyanka Singh, Xue Li, Yang CaoAbstract
Large language models (LLMs) frequently memorize sensitive or personal information, raising significant privacy concerns. Existing variants of differential privacy stochastic gradient descent (DPSGD) inject uniform noise into every gradient step, significantly extending training time and reducing model accuracy. We propose that concentrating noise primarily on gradients associated with sensitive tokens can substantially decrease DP training time, strengthen the protection of sensitive information, and simultaneously preserve the model's performance on non-sensitive data. We operationalize this insight through Adaptive Token-Weighted Differential Privacy (ATDP), a modification of vanilla DP-SGD that adaptively assigns different gradient weights to sensitive and non-sensitive tokens. By employing a larger noise scale at the early stage of training, ATDP rapidly disrupts memorization of sensitive content. As a result, ATDP only requires a few additional epochs of lightweight post-processing following standard fine-tuning, injecting targeted noise primarily on parameters corresponding to sensitive tokens, thus minimally affecting the model's general capabilities. ATDP can be seamlessly integrated into any existing DP-based fine-tuning pipeline or directly applied to non-private models as a fast privacy-enhancing measure. Additionally, combined with an initial redacted fine-tuning phase, ATDP forms a streamlined DP pipeline that achieves comparable canary protection to state-of-the-art DP-SGD methods, significantly reduces the computational overhead of DP fine-tuning, shortening training time by approximately 90 percent, while achieving comparable or superior privacy protection and minimal accuracy degradation.
Abstract (translated)
大型语言模型（LLM）经常记忆敏感或个人信息，引发了严重的隐私问题。现有的差分隐私随机梯度下降（DPSGD）变体在每个梯度步骤中注入均匀噪声，这显著延长了训练时间并降低了模型的准确性。我们提出将噪声主要集中在与敏感标记相关的梯度上可以大幅减少DP训练时间、增强对敏感信息的保护，并同时保持模型在非敏感数据上的性能。 为此，我们提出了自适应令牌加权差分隐私（ATDP），这是对原始DP-SGD的一种改进，在其中根据敏感和非敏感标记自适应地分配不同的梯度权重。通过在训练初期使用更大的噪声尺度，ATDP可以快速中断对敏感内容的记忆过程。因此，ATDP仅需要在标准微调之后进行少量的轻量级后处理，主要针对与敏感标记相关的参数注入定向噪声，从而最小限度地影响模型的一般能力。 ATDP可以无缝整合到任何现有的基于差分隐私（DP）的微调管道中，也可以直接应用于非私有模型以快速增强其隐私。此外，结合初始删减微调阶段，ATDP形成了一条简化的DP流水线，该流程实现了与现有最先进的DP-SGD方法相当的预警保护，并显著减少了DP微调的计算开销，将训练时间缩短约90%，同时达到或优于现有的隐私保护水平和最小化准确性下降。
URL
https://arxiv.org/abs/2509.23246
PDF
-
Coordination Requires Simplification: Thermodynamic Bounds on Multi-Objective Compromise in Natural and Artificial Intelligence
2025-09-27 06:16:56Atma AnandAbstract
Information-processing systems coordinating across multiple agents and objectives face fundamental thermodynamic constraints. We show that solutions with maximum utility to act as coordination focal points have much higher selection pressure for being findable across agents rather than accuracy. We derive that the information-theoretic minimum description length of coordination protocols to precision $\varepsilon$ scales as $L(P)\geq NK\log_2 K+N^2d^2\log (1/\varepsilon)$ for $N$ agents with $d$ potentially conflicting objectives and internal model complexity $K$. This scaling forces progressive simplification, with coordination dynamics changing the environment itself and shifting optimization across hierarchical levels. Moving from established focal points requires re-coordination, creating persistent metastable states and hysteresis until significant environmental shifts trigger phase transitions through spontaneous symmetry breaking. We operationally define coordination temperature to predict critical phenomena and estimate coordination work costs, identifying measurable signatures across systems from neural networks to restaurant bills to bureaucracies. Extending the topological version of Arrow's theorem on the impossibility of consistent preference aggregation, we find it recursively binds whenever preferences are combined. This potentially explains the indefinite cycling in multi-objective gradient descent and alignment faking in Large Language Models trained with reinforcement learning with human feedback. We term this framework Thermodynamic Coordination Theory (TCT), which demonstrates that coordination requires radical information loss.
Abstract (translated)
信息处理系统在多个代理和目标之间进行协调时面临基本的热力学约束。我们展示了具有最大效用的解决方案作为协调焦点，其对跨代理可发现性有更高的选择压力，而非准确性。我们推导出协调协议的信息理论最小描述长度$ L(P) \geq NK\log_2 K + N^2d^2\log (1/\varepsilon)$，其中对于具有$d$个潜在冲突目标和内部模型复杂度为$K$的$N$个代理而言，其精确度达到$\varepsilon$。这种扩展关系迫使系统逐步简化，并且协调动态改变了环境本身，优化层次也随之变化。从已建立的焦点移动需要重新协调，这会创建持久的亚稳态和滞后效应，直到重大的环境变化通过自发对称性破缺触发相变为止。 我们操作性地定义了协调温度来预测临界现象并估算协调工作成本，在神经网络、餐厅账单到官僚机构等系统中可测量出这些标志。我们将阿罗定理（关于偏好聚合一致性不可能性的理论）的拓扑版本扩展，发现当偏好组合时其递归约束存在。这可能解释了多目标梯度下降中的无限循环以及通过人类反馈强化学习训练的大规模语言模型中的对齐伪造现象。 我们称此框架为热力学协调理论(TCT)，它表明协调需要彻底的信息损失。
URL
https://arxiv.org/abs/2509.23144
PDF
-
Teleoperator-Aware and Safety-Critical Adaptive Nonlinear MPC for Shared Autonomy in Obstacle Avoidance of Legged Robots
2025-09-26 18:21:13Ruturaj Sambhus, Muneeb Ahmad, Basit Muhammad Imran, Sujith Vijayan, Dylan P. Losey, Kaveh Akbari HamedAbstract
Ensuring safe and effective collaboration between humans and autonomous legged robots is a fundamental challenge in shared autonomy, particularly for teleoperated systems navigating cluttered environments. Conventional shared-control approaches often rely on fixed blending strategies that fail to capture the dynamics of legged locomotion and may compromise safety. This paper presents a teleoperator-aware, safety-critical, adaptive nonlinear model predictive control (ANMPC) framework for shared autonomy of quadrupedal robots in obstacle-avoidance tasks. The framework employs a fixed arbitration weight between human and robot actions but enhances this scheme by modeling the human input with a noisily rational Boltzmann model, whose parameters are adapted online using a projected gradient descent (PGD) law from observed joystick commands. Safety is enforced through control barrier function (CBF) constraints integrated into a computationally efficient NMPC, ensuring forward invariance of safe sets despite uncertainty in human behavior. The control architecture is hierarchical: a high-level CBF-based ANMPC (10 Hz) generates blended human-robot velocity references, a mid-level dynamics-aware NMPC (60 Hz) enforces reduced-order single rigid body (SRB) dynamics to track these references, and a low-level nonlinear whole-body controller (500 Hz) imposes the full-order dynamics via quadratic programming to track the mid-level trajectories. Extensive numerical and hardware experiments, together with a user study, on a Unitree Go2 quadrupedal robot validate the framework, demonstrating real-time obstacle avoidance, online learning of human intent parameters, and safe teleoperator collaboration.
Abstract (translated)
确保人类与自主腿足机器人之间安全有效的协作是共享自主系统中的一个基本挑战，特别是在处理复杂环境的远程操作系统中。传统共控制方法通常依赖于固定的融合策略，这些策略无法捕捉腿足运动的动力学特性，并可能影响安全性。本文提出了一种基于感知操作员、关注安全性的自适应非线性模型预测控制（ANMPC）框架，用于四足机器人在避障任务中的共享自主驾驶。 该框架采用固定的人机行动仲裁权重，但通过使用嘈杂的理性Boltzmann模型来模拟人的输入，并利用从观测到的操作杆命令中在线调整参数的方法来增强这一方案。安全机制则通过将控制屏障函数（CBF）约束整合到计算效率高的NMPC中实现，在存在人类行为不确定性的情况下仍能确保安全集的前向不变性。控制系统架构是分层设计的：高级别的CBF基础ANMPC（10Hz）生成融合的人机速度参考；中级别的动力学感知NMPC（60Hz）采用简化的单刚体（SRB）动力学模型来跟踪这些参考值；低级别的非线性全身控制器（500Hz）通过二次规划实施完整的动态模型以追踪中间层级的轨迹。 对Unitree Go2四足机器人进行了广泛的数值和硬件实验，以及用户研究，验证了这一框架的有效性。结果表明该系统实现了实时避障、在线学习人类意图参数，并且安全的人机协作。
URL
https://arxiv.org/abs/2509.22815
PDF
