Gradient_Descent
-
From Gradients to Riccati Geometry: Kalman World Models for Single-Pass Learning
2026-03-13 00:30:30Andrew KirulutaAbstract
Backpropagation dominates modern machine learning, yet it is not the only principled method for optimizing dynamical systems. We propose Kalman World Models (KWM), a class of learned state-space models trained via recursive Bayesian filtering rather than reverse-mode automatic differentiation. Instead of gradient descent updates, we replace parameter learning with Kalman-style gain adaptation. Training becomes online filtering; error signals become innovations. We further extend this framework to transformer-based large language models (LLMs), where internal activations are treated as latent dynamical states corrected via innovation terms. This yields a gradient-free training and adaptation paradigm grounded in control theory. We derive stability conditions, analyze computational complexity, and provide empirical results on sequence modeling tasks demonstrating competitive performance with improved robustness and continual adaptation properties.
Abstract (translated)
URL
https://arxiv.org/abs/2603.13423
PDF
-
Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights
2026-03-12 17:49:30Yulu Gan, Phillip IsolaAbstract
Pretraining produces a learned parameter vector that is typically treated as a starting point for further iterative adaptation. In this work, we instead view the outcome of pretraining as a distribution over parameter vectors, whose support already contains task-specific experts. We show that in small models such expert solutions occupy a negligible fraction of the volume of this distribution, making their discovery reliant on structured optimization methods such as gradient descent. In contrast, in large, well-pretrained models the density of task-experts increases dramatically, so that diverse, task-improving specialists populate a substantial fraction of the neighborhood around the pretrained weights. Motivated by this perspective, we explore a simple, fully parallel post-training method that samples $N$ parameter perturbations at random, selects the top $K$, and ensembles predictions via majority vote. Despite its simplicity, this approach is competitive with standard post-training methods such as PPO, GRPO, and ES for contemporary large-scale models.
Abstract (translated)
预训练产生的参数向量通常被视作进一步迭代适应的起点。然而，在这项工作中，我们将预训练的结果视为参数向量上的分布，该分布的支持集中已包含了特定任务的专家模型。我们展示了在小规模模型中，这些专家解决方案所占的空间非常微小，因此其发现依赖于结构化的优化方法如梯度下降法。相比之下，在大规模且充分预训练的模型中，任务相关专家的数量显著增加，多样而有效的专家占据了预训练权重附近很大一部分区域。 基于这一视角，我们探索了一种简单的、完全并行的后训练方法：随机抽取$N$个参数扰动，选择其中前$K$个，并通过多数投票的方式进行集成预测。尽管该方法简单，它在当代大规模模型中的表现仍可与标准的后训练方法（如PPO, GRPO和ES）相媲美。
URL
https://arxiv.org/abs/2603.12228
PDF
-
Continual Learning with Vision-Language Models via Semantic-Geometry Preservation
2026-03-12 15:25:53Chiyuan He, Zihuan Qiu, Fanman Meng, Runtong Zhang, Linfeng Xu, Qingbo Wu, Hongliang LiAbstract
Continual learning of pretrained vision-language models (VLMs) is prone to catastrophic forgetting, yet current approaches adapt to new tasks without explicitly preserving the cross-modal semantic geometry inherited from pretraining and previous stages, allowing new-task supervision to induce geometric distortion. We observe that the most pronounced drift tends to concentrate in vulnerable neighborhoods near the old-new semantic interface, where shared visual patterns are easily re-explained by new textual semantics. To address this under an exemplar-free constraint, we propose Semantic Geometry Preservation for Continual Learning (SeGP-CL). SeGP-CL first probes the drift-prone region by constructing a compact set of adversarial anchors with dual-targeted projected gradient descent (DPGD), which drives selected new-task seeds toward old-class semantics while remaining faithful in raw visual space. During training, we preserve cross-modal structure by anchor-guided cross-modal geometry distillation (ACGD), and stabilize the textual reference frame across tasks via a lightweight text semantic-geometry regularization (TSGR). After training, we estimate anchor-induced raw-space drift to transfer old visual prototypes and perform dual-path inference by fusing cross-modal and visual cues. Extensive experiments on five continual learning benchmarks demonstrate that SeGP-CL consistently improves stability and forward transfer, achieving state-of-the-art performance while better preserving semantic geometry of VLMs.
Abstract (translated)
持续学习预训练的视觉语言模型（VLM）容易出现灾难性遗忘，而当前的方法在适应新任务时没有显式地保留从预训练和先前阶段继承来的跨模态语义几何结构，这使得新的监督可能导致几何畸变。我们观察到最明显的漂移往往集中在旧任务与新任务之间的脆弱邻域中，在这些区域共享的视觉模式很容易被新的文本语义重新解释。 为了在没有示例的情况下解决这个问题，我们提出了用于持续学习的语义几何保持（Semantic Geometry Preservation for Continual Learning, SeGP-CL）。SeGP-CL 首先通过使用双重目标投影梯度下降法 (DPGD) 构建一组紧凑的对抗性锚点来探测易漂移区域。这种方法引导新任务种子向着旧类语义移动，同时保持在原始视觉空间中的忠实度。 在训练过程中，我们利用锚点指导下的跨模态几何知识蒸馏（ACGD）保留了跨模态结构，并通过轻量级的文本语义几何正则化（TSGR）在整个任务中稳定文本参考框架。经过训练后，我们估计由锚点引起的原始空间漂移来传输旧视觉原型，并进行双路径推理以融合跨模态和视觉线索。 在五个持续学习基准上的广泛实验表明，SeGP-CL 一致地提高了稳定性和前向迁移能力，在保持 VLM 的语义几何结构的同时实现了最先进的性能。
URL
https://arxiv.org/abs/2603.12055
PDF
-
Taming Vision Priors for Data Efficient mmWave Channel Modeling
2026-03-11 04:35:51Zhenlin An, Longfei Shangguan, John Kaewell, Philip Pietraski, Jelena Senic, Camillo Gentile, Nada Golmie, Kyle JamiesonAbstract
Accurately modeling millimeter-wave (mmWave) propagation is essential for real-time AR and autonomous systems. Differentiable ray tracing offers a physics-grounded solution but still facing deployment challenges due to its over-reliance on exhaustive channel measurements or brittle, hand-tuned scene models for material properties. We present VisRFTwin, a scalable and data-efficient digital-twin framework that integrates vision-derived material priors with differentiable ray tracing. Multi-view images from commodity cameras are processed by a frozen Vision-Language Model to extract dense semantic embeddings, which are translated into initial estimates of permittivity and conductivity for scene surfaces. These priors initialize a Sionna-based differentiable ray tracer, which rapidly calibrates material parameters via gradient descent with only a few dozen sparse channel soundings. Once calibrated, the association between vision features and material parameters is retained, enabling fast transfer to new scenarios without repeated calibration. Evaluations across three real-world scenarios, including office interiors, urban canyons, and dynamic public spaces show that VisRFTwin reduces channel measurement needs by up to 10$\times$ while achieving a 59% lower median delay spread error than pure data-driven deep learning methods.
Abstract (translated)
URL
https://arxiv.org/abs/2603.13383
PDF
-
On the Learning Dynamics of Two-layer Linear Networks with Label Noise SGD
2026-03-11 04:26:54Tongcheng Zhang, Zhanpeng Zhou, Mingze Wang, Andi Han, Wei Huang, Taiji Suzuki, Junchi YanAbstract
One crucial factor behind the success of deep learning lies in the implicit bias induced by noise inherent in gradient-based training algorithms. Motivated by empirical observations that training with noisy labels improves model generalization, we delve into the underlying mechanisms behind stochastic gradient descent (SGD) with label noise. Focusing on a two-layer over-parameterized linear network, we analyze the learning dynamics of label noise SGD, unveiling a two-phase learning behavior. In \emph{Phase I}, the magnitudes of model weights progressively diminish, and the model escapes the lazy regime; enters the rich regime. In \emph{Phase II}, the alignment between model weights and the ground-truth interpolator increases, and the model eventually converges. Our analysis highlights the critical role of label noise in driving the transition from the lazy to the rich regime and minimally explains its empirical success. Furthermore, we extend these insights to Sharpness-Aware Minimization (SAM), showing that the principles governing label noise SGD also apply to broader optimization algorithms. Extensive experiments, conducted under both synthetic and real-world setups, strongly support our theory. Our code is released at this https URL.
Abstract (translated)
深度学习成功的一个关键因素在于基于梯度的训练算法中固有的噪声所引起的隐式偏差。受使用带噪标签进行训练可以提高模型泛化能力这一经验观察的启发，我们深入研究了带有标签噪声的随机梯度下降（SGD）背后的机制。重点分析了一个两层过参数化的线性网络的学习动态，在这个过程中揭示了标签噪声SGD的双阶段学习行为。 在**第一阶段（Phase I）**中，模型权重的大小逐渐减小，并且模型从懒惰模式逃逸进入丰富模式。 在**第二阶段（Phase II）**中，模型权重与真实插值器之间的对齐度增加，最终达到收敛状态。 我们的分析强调了标签噪声在推动从懒惰模式向丰富模式转变中的关键作用，并至少部分解释了其经验上的成功。此外，我们将这些见解扩展到了Sharpness-Aware Minimization（SAM）中，表明指导带有标签噪声的SGD的原则也适用于更广泛的优化算法。我们通过在合成和真实世界设置下进行的一系列实验强烈支持我们的理论。我们的代码可在[此链接](https://this https URL)处获取。
URL
https://arxiv.org/abs/2603.10397
PDF
-
ReMix: Reinforcement routing for mixtures of LoRAs in LLM finetuning
2026-03-10 18:51:27Ruizhong Qiu, Hanqing Zeng, Yinglong Xia, Yiwen Meng, Ren Chen, Jiarui Feng, Dongqi Fu, Qifan Wang, Jiayi Liu, Jun Xiao, Xiangjun Fan, Benyu Zhang, Hong Li, Zhining Liu, Hyunsik Yoo, Zhichen Zeng, Tianxin Wei, Hanghang TongAbstract
Low-rank adapters (LoRAs) are a parameter-efficient finetuning technique that injects trainable low-rank matrices into pretrained models to adapt them to new tasks. Mixture-of-LoRAs models expand neural networks efficiently by routing each layer input to a small subset of specialized LoRAs of the layer. Existing Mixture-of-LoRAs routers assign a learned routing weight to each LoRA to enable end-to-end training of the router. Despite their empirical promise, we observe that the routing weights are typically extremely imbalanced across LoRAs in practice, where only one or two LoRAs often dominate the routing weights. This essentially limits the number of effective LoRAs and thus severely hinders the expressive power of existing Mixture-of-LoRAs models. In this work, we attribute this weakness to the nature of learnable routing weights and rethink the fundamental design of the router. To address this critical issue, we propose a new router designed that we call Reinforcement Routing for Mixture-of-LoRAs (ReMix). Our key idea is using non-learnable routing weights to ensure all active LoRAs to be equally effective, with no LoRA dominating the routing weights. However, our routers cannot be trained directly via gradient descent due to our non-learnable routing weights. Hence, we further propose an unbiased gradient estimator for the router by employing the reinforce leave-one-out (RLOO) technique, where we regard the supervision loss as the reward and the router as the policy in reinforcement learning. Our gradient estimator also enables to scale up training compute to boost the predictive performance of our ReMix. Extensive experiments demonstrate that our proposed ReMix significantly outperform state-of-the-art parameter-efficient finetuning methods under a comparable number of activated parameters.
Abstract (translated)
低秩适配器（LoRAs）是一种参数高效的微调技术，它通过将可训练的低秩矩阵注入预训练模型来使这些模型适应新任务。Mixture-of-LoRAs模型通过将每一层的输入路由到该层的一小部分专门化的LoRA中，高效地扩展神经网络。现有的Mixture-of-LoRAs路由器为每个LoRA分配一个可学习的路由权重，以实现端到端训练。 尽管这种方法在实践中表现出了潜力，但我们观察到，在实际应用中，这些路由权重通常会在不同LoRA之间极为不平衡，其中只有少数几个LoRA会主导整个路由权重。这实质上限制了有效LoRA的数量，从而严重阻碍了现有Mixture-of-LoRAs模型的表达能力。 为了解决这一关键问题，我们归因于可学习路由权重的本质，并重新审视路由器的基本设计。为此，我们提出了一种新的路由器设计方案，称为针对混合低秩适配器（Reinforcement Routing for Mixture-of-LoRAs, ReMix）。我们的主要思想是使用不可学习的路由权重来确保所有激活状态下的LoRA同等有效，没有单一的LoRA能够主导整个路由权重。 然而，由于我们使用的非可学习路由权重，直接通过梯度下降训练这种路由器是不可能的。因此，我们进一步提出了一种无偏导数估计器，通过采用强化学习中的Reinforce Leave-One-Out (RLOO)技术来实现，将监督损失视为奖励，而路由器则作为策略。我们的导数估计器还使计算资源能够扩展，从而提升我们ReMix的预测性能。 广泛的实验结果表明，在激活参数数量大致相等的情况下，所提出的ReMix方法显著优于现有的基于参数效率的微调方法。
URL
https://arxiv.org/abs/2603.10160
PDF
-
Stein Variational Ergodic Surface Coverage with SE Constraints
2026-03-10 10:13:36Jiayun Li, Yufeng Jin, Sangli Teng, Dejian Gong, Georgia ChalvatzakiAbstract
Surface manipulation tasks require robots to generate trajectories that comprehensively cover complex 3D surfaces while maintaining precise end-effector poses. Existing ergodic trajectory optimization (TO) methods demonstrate success in coverage tasks, while struggling with point-cloud targets due to the nonconvex optimization landscapes and the inadequate handling of SE(3) constraints in sampling-as-optimization (SAO) techniques. In this work, we introduce a preconditioned SE(3) Stein Variational Gradient Descent (SVGD) approach for SAO ergodic trajectory generation. Our proposed approach comprises multiple innovations. First, we reformulate point-cloud ergodic coverage as a manifold-aware sampling problem. Second, we derive SE(3)-specific SVGD particle updates, and, third, we develop a preconditioner to accelerate TO convergence. Our sampling-based framework consistently identifies superior local optima compared to strong optimization-based and SAO baselines while preserving the SE(3) geometric structure. Experiments on a 3D point-cloud surface coverage benchmark and robotic surface drawing tasks demonstrate that our method achieves superior coverage quality with tractable computation in our setting relative to existing TO and SAO approaches, and is validated in real-world robot experiments.
Abstract (translated)
表面操作任务要求机器人生成全面覆盖复杂三维曲面的轨迹，同时保持末端执行器的精确姿态。现有的遍历轨迹优化（TO）方法在覆盖率任务中表现出色，但在处理点云目标时由于非凸优化景观和采样-优化(SAO)技术对SE(3)约束处理不足而遇到困难。在这项工作中，我们引入了一种预条件化的SE(3) Stein变分梯度下降法（SVGD）用于SAO遍历轨迹生成。我们的方法包含了多项创新：首先，我们将点云遍历覆盖重构为一个关注流形的采样问题；其次，我们推导了适用于SE(3)特性的SVGD粒子更新公式；第三，我们开发了一种预处理器以加速TO收敛性。 基于采样的框架在识别局部最优解方面优于强大的优化基线和SAO方法，同时保留了SE(3)几何结构。实验表明，在一个三维点云表面覆盖基准测试和机器人表面绘图任务中，相较于现有的TO和SAO方法，我们的方法能够在现实世界的机器人实验中实现更高质量的覆盖率，并且计算量是可接受的。
URL
https://arxiv.org/abs/2603.09458
PDF
-
Minor First, Major Last: A Depth-Induced Implicit Bias of Sharpness-Aware Minimization
2026-03-09 12:09:14Chaewon Moon, Dongkuk Si, Chulhee YunAbstract
We study the implicit bias of Sharpness-Aware Minimization (SAM) when training $L$-layer linear diagonal networks on linearly separable binary classification. For linear models ($L=1$), both $\ell_\infty$- and $\ell_2$-SAM recover the $\ell_2$ max-margin classifier, matching gradient descent (GD). However, for depth $L = 2$, the behavior changes drastically -- even on a single-example dataset. For $\ell_\infty$-SAM, the limit direction depends critically on initialization and can converge to $\mathbf{0}$ or to any standard basis vector, in stark contrast to GD, whose limit aligns with the basis vector of the dominant data coordinate. For $\ell_2$-SAM, we show that although its limit direction matches the $\ell_1$ max-margin solution as in the case of GD, its finite-time dynamics exhibit a phenomenon we call "sequential feature amplification", in which the predictor initially relies on minor coordinates and gradually shifts to larger ones as training proceeds or initialization increases. Our theoretical analysis attributes this phenomenon to $\ell_2$-SAM's gradient normalization factor applied in its perturbation, which amplifies minor coordinates early and allows major ones to dominate later, giving a concrete example where infinite-time implicit-bias analyses are insufficient. Synthetic and real-data experiments corroborate our findings.
Abstract (translated)
我们在研究训练$L$层线性对角网络（用于线性可分二元分类）时，Sharpness-Aware Minimization (SAM) 的隐式偏差。对于线性模型($L=1$)，不论是$\ell_\infty$-SAM 还是 $\ell_2$-SAM 都恢复了 $\ell_2$ 最大间隔分类器，这与梯度下降（GD）一致。然而，当深度$L = 2$时，行为有了巨大的变化——甚至在单一示例数据集上也是如此。 对于$\ell_\infty$-SAM，极限方向在很大程度上依赖于初始化，并且可以收敛到零向量或任何标准基向量，这与GD形成鲜明对比，GD的极限则与主导数据坐标的基向量对齐。而对于$\ell_2$-SAM，我们证明尽管其极限方向匹配了梯度下降情况下 $\ell_1$ 最大间隔解的情况，但其有限时间动态表现出一种现象我们称之为“顺序特征放大”，即预测器最初依赖于次要坐标，并随着训练的进行或初始化的增加而逐渐转向主要坐标。我们的理论分析认为这种现象归因于$\ell_2$-SAM 在其扰动中应用的梯度标准化因子，这在早期放大了次要坐标并在后期允许主要坐标占据主导地位，从而提供了一个具体例子说明无限时间隐式偏差分析是不够的。 合成数据和实际数据实验验证了我们的发现。
URL
https://arxiv.org/abs/2603.08290
PDF
-
Norm-Hierarchy Transitions in Representation Learning: When and Why Neural Networks Abandon Shortcuts
2026-03-07 20:07:16Truong Xuan Khanh, Truong Quynh HoaAbstract
Neural networks often rely on spurious shortcuts for many epochs before discovering structured representations. However, the mechanism governing when this transition occurs and whether its timing can be predicted remains unclear. Prior work shows that gradient descent converges to low norm solutions and that neural networks exhibit simplicity bias, but neither explains the timescale of the transition from shortcut features to structured representations. We introduce the Norm-Hierarchy Transition (NHT) framework, which explains delayed representation learning as the slow traversal of a hierarchy of parameter norms during regularized optimization. When multiple interpolating solutions exist with different norms, weight decay gradually moves the model from high norm shortcut solutions toward lower norm structured representations. We derive a tight bound showing that the transition delay grows logarithmically with the ratio between shortcut and structured norms. Experiments on modular arithmetic, CIFAR-10 with spurious features, CelebA, and Waterbirds support the predictions of the framework. The results suggest that grokking, shortcut learning, and delayed feature discovery arise from a common mechanism based on norm hierarchy traversal during training.
Abstract (translated)
神经网络常常在发现结构化表示之前依赖于伪捷径（即无关紧要的特征）很多个周期。然而，决定这一转换何时发生以及其时间是否可以预测的机制仍然不清楚。先前的工作表明，梯度下降收敛到低范数解，并且神经网络表现出简洁性偏见，但这两者都不能解释从伪特征到结构化表示的时间尺度上的过渡过程。 我们引入了基于规范层级转换（Norm-Hierarchy Transition, NHT）框架，该框架将延迟的表示学习解释为在正则化的优化过程中缓慢遍历参数范数层级的过程。当存在多个具有不同范数的插值解时，权重衰减会逐渐地将模型从高范数的伪特征解决方案引导至低范数的结构化表征。 我们推导出一个紧致边界，表明过渡延迟随捷径和结构化范数之间的比率呈对数增长。在模运算、具有伪特征的CIFAR-10、CelebA以及Waterbirds数据集上的实验均验证了该框架的预测结果。 这些研究结果暗示了“理解”（grokking）、捷径学习及延迟特征发现现象源自训练过程中基于规范层级遍历的共同机制。
URL
https://arxiv.org/abs/2603.07323
PDF
-
NePPO: Near-Potential Policy Optimization for General-Sum Multi-Agent Reinforcement Learning
2026-03-07 01:35:52Addison Kalanther, Sanika Bharvirkar, Shankar Sastry, Chinmay MaheshwariAbstract
Multi-agent reinforcement learning (MARL) is increasingly used to design learning-enabled agents that interact in shared environments. However, training MARL algorithms in general-sum games remains challenging: learning dynamics can become unstable, and convergence guarantees typically hold only in restricted settings such as two-player zero-sum or fully cooperative games. Moreover, when agents have heterogeneous and potentially conflicting preferences, it is unclear what system-level objective should guide learning. In this paper, we propose a new MARL pipeline called Near-Potential Policy Optimization (NePPO) for computing approximate Nash equilibria in mixed cooperative--competitive environments. The core idea is to learn a player-independent potential function such that the Nash equilibrium of a cooperative game with this potential as the common utility approximates a Nash equilibrium of the original game. To this end, we introduce a novel MARL objective such that minimizing this objective yields the best possible potential function candidate and consequently an approximate Nash equilibrium of the original game. We develop an algorithmic pipeline that minimizes this objective using zeroth-order gradient descent and returns an approximate Nash equilibrium policy. We empirically show the superior performance of this approach compared to popular baselines such as MAPPO, IPPO and MADDPG.
Abstract (translated)
多智能体强化学习（MARL）被广泛用于设计能够在共享环境中互动的学习型代理。然而，在总和博弈中训练MARL算法仍然存在挑战：学习动态可能会变得不稳定，且收敛保证通常仅在受限情况下有效，如两人零和游戏或完全合作的游戏。此外，当代理具有异质且可能冲突的偏好时，尚不清楚应使用何种系统级目标来指导学习过程。 本文提出了一种新的MARL流水线——近似势函数策略优化（NePPO），用于计算混合合作-竞争环境中的近似纳什均衡。核心思想是学习一种独立于玩家的潜在函数，使得具有该潜在函数作为公共效用的合作博弈的纳什均衡能够接近原始博弈的近似纳什均衡。 为此，我们引入了一种新的MARL目标函数，通过最小化这个目标函数可以获得最佳的势函数候选，并进而得出原游戏的一个近似纳什均衡策略。我们开发了一个算法流水线，使用零阶梯度下降来最小化该目标函数，并返回一个近似纳什均衡策略。 实验结果表明，与流行的基准方法（如MAPPO、IPPO和MADDPG）相比，这种方法在性能上表现出显著优势。
URL
https://arxiv.org/abs/2603.06977
PDF
-
COLD-Steer: Steering Large Language Models via In-Context One-step Learning Dynamics
2026-03-06 17:27:27Kartik Sharma, Rakshit S. TrivediAbstract
Activation steering methods enable inference-time control of large language model (LLM) behavior without retraining, but current approaches face a fundamental trade-off: sample-efficient methods suboptimally capture steering signals from labeled examples, while methods that better extract these signals require hundreds to thousands of examples. We introduce COLD-Steer, a training-free framework that steers LLM activations by approximating the representational changes that would result from gradient descent on in-context examples. Our key insight is that the effect of fine-tuning on a small set of examples can be efficiently approximated at inference time without actual parameter updates. We formalize this through two complementary approaches: (i) a unit kernel approximation method that updates the activations directly using gradients with respect to them, normalized across examples, and (ii) a finite-difference approximation requiring only two forward passes regardless of example count. Experiments across a variety of steering tasks and benchmarks demonstrate that COLD-Steer achieves upto 95% steering effectiveness while using 50 times fewer samples compared to the best baseline. COLD-Steer facilitates accommodating diverse perspectives without extensive demonstration data, which we validate through our experiments on pluralistic alignment tasks. Our framework opens new possibilities for adaptive, context-aware model control that can flexibly address varying loss-driven human preferences through principled approximation of learning dynamics rather than specialized training procedures.
Abstract (translated)
激活引导方法能够在不重新训练的情况下控制大规模语言模型（LLM）的行为，但当前的方法面临一个根本性的权衡：样本效率高的方法无法充分捕捉标记示例中的引导信号，而能够更好地提取这些信号的方法则需要数百到数千个样本。我们提出了COLD-Steer框架，这是一个无需训练的框架，通过近似在上下文示例上进行梯度下降所产生的表示变化来引导LLM激活状态。我们的关键见解在于，在推理时可以通过不实际更新参数的方式高效地模拟针对少量示例微调的效果。 为了实现这一点，我们提出了两种互补的方法：（i）一种单位核近似方法，这种方法直接使用关于这些激活的梯度进行更新，并且在各个样本之间进行了归一化处理；（ii）有限差分近似法仅需要两次前向传递，无论示例数量多少。一系列针对不同引导任务和基准测试的实验表明，COLD-Steer能够在使用比最佳基线少50倍样本的情况下实现高达95%的引导有效性。 通过我们在多样视角一致性的任务上的实验验证了COLD-Steer框架能够促进在无需大量示例数据的情况下适应多元观点。我们的框架为灵活地应对由损失驱动的人类偏好提供了新的可能性，它通过合理近似学习动态而不是专门训练程序来实现这一点，从而开启了自适应的、上下文感知模型控制的新途径。
URL
https://arxiv.org/abs/2603.06495
PDF
-
STL-SVPIO: Signal Temporal Logic guided Stein Variational Path Integral Optimization
2026-03-06 00:53:38Hongrui Zheng, Zirui Zang, Ahmad Amine, Cristian Ioan Vasile, Rahul MangharamAbstract
Signal Temporal Logic (STL) enables formal specification of complex spatiotemporal constraints for robotic task planning. However, synthesizing long-horizon continuous control trajectories from complex STL specifications is fundamentally challenging due to the nested structure of STL robustness objectives. Existing solver-based methods, such as Mixed-Integer Linear Programming (MILP), suffer from exponential scaling, whereas sampling methods, such as Model-Predictive Path Integral control (MPPI), struggle with sparse, long-horizon costs. We introduce Signal Temporal Logic guided Stein Variational Path Integral Optimization (STL-SVPIO), which reframes STL as a globally informative, differentiable reward-shaping mechanism. By leveraging Stein Variational Gradient Descent and differentiable physics engines, STL-SVPIO transports a mutually repulsive swarm of control particles toward high robustness regions. Our method transforms sparse logical satisfaction into tractable variational inference, mitigating the severe local minima traps of standard gradient-based methods. We demonstrate that STL-SVPIO significantly outperforms existing methods in both robustness and efficiency for traditional STL tasks. Moreover, it solves complex long-horizon tasks, including multi-agent coordination with synchronization and queuing while baselines either fail to discover feasible solutions, or become computationally intractable. Finally, we use STL-SVPIO in agile robotic motion planning tasks with nonlinear dynamics, such as 7-DoF manipulation and half cheetah back flips to show the generalizability of our algorithm.
Abstract (translated)
URL
https://arxiv.org/abs/2603.13333
PDF
-
FedBCD:Communication-Efficient Accelerated Block Coordinate Gradient Descent for Federated Learning
2026-03-05 12:37:17Junkang Liu, Fanhua Shang, Yuanyuan Liu, Hongying Liu, Yuangang Li, YunXiang GongAbstract
Although Federated Learning has been widely studied in recent years, there are still high overhead expenses in each communication round for large-scale models such as Vision Transformer. To lower the communication complexity, we propose a novel Federated Block Coordinate Gradient Descent (FedBCGD) method for communication efficiency. The proposed method splits model parameters into several blocks, including a shared block and enables uploading a specific parameter block by each client, which can significantly reduce communication overhead. Moreover, we also develop an accelerated FedBCGD algorithm (called FedBCGD+) with client drift control and stochastic variance reduction. To the best of our knowledge, this paper is the first work on parameter block communication for training large-scale deep models. We also provide the convergence analysis for the proposed algorithms. Our theoretical results show that the communication complexities of our algorithms are a factor $1/N$ lower than those of existing methods, where $N$ is the number of parameter blocks, and they enjoy much faster convergence than their counterparts. Empirical results indicate the superiority of the proposed algorithms compared to state-of-the-art algorithms. The code is available at this https URL.
Abstract (translated)
尽管联邦学习（Federated Learning）近年来得到了广泛的研究，但对于大规模模型如视觉变换器（Vision Transformer），每次通信轮次中仍然存在高昂的开销。为了降低通信复杂性，我们提出了一种新的、旨在提高通信效率的联邦块坐标梯度下降法（FedBCGD）。该方法将模型参数分割成若干块，包括一个共享块，并允许每个客户端上传特定参数块，这可以显著减少通信开销。此外，我们还开发了一种加速版FedBCGD算法（称为FedBCGD+），它结合了客户端漂移控制和随机方差减小技术。 据我们所知，本文是关于用于训练大规模深度模型的参数块通信的第一项工作。我们也提供了对所提出算法的收敛性分析。我们的理论结果表明，与现有方法相比，我们算法的通信复杂度低了一个因子$1/N$（其中N为参数块的数量），并且它们比现有的方法具有更快的收敛速度。实验结果显示了所提出的算法相较于最先进的算法的优势。代码可在提供的链接中获取：[此处应插入实际URL]。
URL
https://arxiv.org/abs/2603.05116
PDF
-
Graph Hopfield Networks: Energy-Based Node Classification with Associative Memory
2026-03-03 19:21:05Abinav Rao, Alex Wa, Rishi AthavaleAbstract
We introduce Graph Hopfield Networks, whose energy function couples associative memory retrieval with graph Laplacian smoothing for node classification. Gradient descent on this joint energy yields an iterative update interleaving Hopfield retrieval with Laplacian propagation. Memory retrieval provides regime-dependent benefits: up to 2.0~pp on sparse citation networks and up to 5 pp additional robustness under feature masking; the iterative energy-descent architecture itself is a strong inductive bias, with all variants (including the memory-disabled NoMem ablation) outperforming standard baselines on Amazon co-purchase graphs. Tuning enables graph sharpening for heterophilous benchmarks without architectural changes.
Abstract (translated)
我们介绍了图Hopfield网络，其能量函数将联想记忆检索与图拉普拉斯平滑相结合用于节点分类。在该联合能量上的梯度下降会产生一个迭代更新过程，该过程交替进行霍普菲尔德检索和拉普拉斯传播。记忆检索提供了依赖于具体场景的好处：对于稀疏的引文网络最多可提高2.0个百分点，在特征掩蔽下的额外鲁棒性最高可达5个百分点；这种迭代能量下降架构本身是一个强大的归纳偏置，所有变体（包括禁用内存的NoMem消融）在亚马逊共购图上均优于标准基线。调整可以无需架构更改即可实现异ophilous基准测试中的图形锐化。
URL
https://arxiv.org/abs/2603.03464
PDF
-
PrivMedChat: End-to-End Differentially Private RLHF for Medical Dialogue Systems
2026-03-03 14:53:20Sudip BhujelAbstract
Large language models are increasingly used for patient-facing medical assistance and clinical decision support, but adapting them to clinical dialogue often requires supervision derived from doctor-patient conversations that may contain sensitive information. Conventional supervised fine-tuning and reinforcement learning from human feedback (RLHF) can amplify memorization risks, enabling empirical membership inference and extraction of rare training-set content. We present PrivMedChat, an end-to-end framework for differentially private RLHF (DP-RLHF) for medical dialogue. Our design enforces differential privacy at every training stage that directly accesses dialogue-derived supervision: (i) Differential Private Stochastic Gradient Descent (DP-SGD) for medical SFT and (ii) DP-SGD for reward model learning from preference pairs. To limit additional privacy expenditure during alignment, we apply DP-SGD to the PPO actor and critic when operating on dialogue-derived prompts, while the reward model remains fixed after DP training. We also introduce an annotation-free preference construction strategy that pairs physician responses with filtered non-expert generations to produce scalable preference data without clinician labeling. Experiments on medical dialogue benchmarks show that PrivMedChat at $\varepsilon=7$ achieves the highest ROUGE-L of 0.156 among all DP models, reduces clinical hallucinations to 1.4% and harmful advice to 0.4%, and obtains the highest overall score of 2.86 in a 3-model LLM-jury evaluation, while producing membership-inference signals that are near chance (AUC 0.510-0.555). We open-source our code at this https URL.
Abstract (translated)
大型语言模型在患者医疗服务和临床决策支持中的应用日益广泛，但要将这些模型适应于医疗对话通常需要基于医生与患者交流的监督数据，而这些数据可能包含敏感信息。传统的有监督微调（Supervised Fine-Tuning, SFT）和从人类反馈中进行强化学习（Reinforcement Learning from Human Feedback, RLHF）可能会放大记忆风险，从而使得经验性的成员推断成为可能，并且可以提取罕见的训练集内容。我们提出了PrivMedChat，这是一个用于医疗对话的差分隐私RLHF (DP-RLHF) 的端到端框架。我们的设计强制在每个直接访问从对话中获取监督数据的训练阶段实施差分隐私：(i) 对于医学SFT使用差分私有随机梯度下降（Differential Private Stochastic Gradient Descent, DP-SGD）；(ii) 在从偏好对学习奖励模型时也采用DP-SGD。为了在调整模型行为时限制额外的隐私开销，当处理由对话生成的提示时，我们只应用DP-SGD于PPO策略和价值函数（actor和critic），而经过差分隐私训练后的奖励模型则保持不变。此外，我们还提出了一种无需标注的偏好构建策略，该策略通过将医生的回答与过滤掉专家部分之后的非专家生成进行配对来创建可扩展且不需要临床医生标记的数据。 在医疗对话基准测试中的实验表明，在$\varepsilon=7$的情况下，PrivMedChat实现了所有差分隐私模型中最高的ROUGE-L值0.156，将临床假阳性率降低到1.4%，有害建议减少至0.4%，并在包含三个大型语言模型的评估中获得了最高总评分2.86。同时，在成员推断信号方面几乎接近随机（AUC 0.510-0.555）。我们的代码可以在这里访问：[此https URL]。 这种框架不仅保护了训练数据中的隐私信息，还在生成医疗服务对话时显著减少了潜在的临床风险和有害建议的产生。
URL
https://arxiv.org/abs/2603.03054
PDF
-
3D Field of Junctions: A Noise-Robust, Training-Free Structural Prior for Volumetric Inverse Problems
2026-03-02 18:11:59Namhoon Kim, Narges Moeini, Justin Romberg, Sara Fridovich-KeilAbstract
Volume denoising is a foundational problem in computational imaging, as many 3D imaging inverse problems face high levels of measurement noise. Inspired by the strong 2D image denoising properties of Field of Junctions (ICCV 2021), we propose a novel, fully volumetric 3D Field of Junctions (3D FoJ) representation that optimizes a junction of 3D wedges that best explain each 3D patch of a full volume, while encouraging consistency between overlapping patches. In addition to direct volume denoising, we leverage our 3D FoJ representation as a structural prior that: (i) requires no training data, and thus precludes the risk of hallucination, (ii) preserves and enhances sharp edge and corner structures in 3D, even under low signal to noise ratio (SNR), and (iii) can be used as a drop-in denoising representation via projected or proximal gradient descent for any volumetric inverse problem with low SNR. We demonstrate successful volume reconstruction and denoising with 3D FoJ across three diverse 3D imaging tasks with low-SNR measurements: low-dose X-ray computed tomography (CT), cryogenic electron tomography (cryo-ET), and denoising point clouds such as those from lidar in adverse weather. Across these challenging low-SNR volumetric imaging problems, 3D FoJ outperforms a mixture of classical and neural methods.
Abstract (translated)
体积去噪是计算成像中的一个基础问题，因为许多三维成像逆向问题是高度测量噪声的。受图像领域节点（ICCV 2021）在二维图像去噪方面的强大性能启发，我们提出了一种全新的、完全体素化的3D图像节点场（3D FoJ）表示方法。这种表示法优化了一个最佳解释全体积中每个三维补丁的3D楔形连接点，并鼓励重叠补丁之间的一致性。除了直接进行体积去噪之外，我们的3D FoJ表示还可以作为结构先验：(i) 不需要训练数据，从而消除了幻觉的风险；(ii) 即使在低信噪比（SNR）条件下也能保持并增强三维中的锐边和角点结构；(iii) 可以通过投影或近似梯度下降用于任何具有低SNR的体素逆向问题去噪。我们在三个不同且测量信噪比较低的3D成像任务中成功展示了3D FoJ的体积重建与降噪：低剂量X射线计算机断层扫描（CT）、冷冻电子断层扫描（cryo-ET）以及恶劣天气下激光雷达点云的去噪。在这些具有挑战性的低SNR体素成像问题上，3D FoJ的表现超过了经典方法和神经网络方法的混合使用。
URL
https://arxiv.org/abs/2603.02149
PDF
-
On the Rate of Convergence of GD in Non-linear Neural Networks: An Adversarial Robustness Perspective
2026-03-02 17:13:33Guy Smorodinsky, Sveta Gimpleson, Itay SafranAbstract
We study the convergence dynamics of Gradient Descent (GD) in a minimal binary classification setting, consisting of a two-neuron ReLU network and two training instances. We prove that even under these strong simplifying assumptions, while GD successfully converges to an optimal robustness margin, effectively maximizing the distance between the decision boundary and the training points, this convergence occurs at a prohibitively slow rate, scaling strictly as $\Theta(1/\ln(t))$. To the best of our knowledge, this establishes the first explicit lower bound on the convergence rate of the robustness margin in a non-linear model. Through empirical simulations, we further demonstrate that this inherent failure mode is pervasive, exhibiting the exact same tight convergence rate across multiple natural network initializations. Our theoretical guarantees are derived via a rigorous analysis of the GD trajectories across the distinct activation patterns of the model. Specifically, we develop tight control over the system's dynamics to bound the trajectory of the decision boundary, overcoming the primary technical challenge introduced by the non-linear nature of the architecture.
Abstract (translated)
我们在一个最小的二元分类设置中研究梯度下降（GD）的收敛动力学，该设置由一个具有两个训练实例的两神经元ReLU网络组成。我们证明，在这些强烈的简化假设下，即使如此，GD仍然成功地以最优鲁棒性边界收敛，并有效最大化决策边界与训练点之间的距离。然而，这种收敛是以极其缓慢的速度发生的，其速度严格遵循$\Theta(1/\ln(t))$的规模。据我们所知，这是首次在非线性模型中明确建立的关于鲁棒性边界的收敛速率下限。 通过经验模拟，我们进一步展示了这一内在失败模式具有普遍性，在多个自然网络初始化的情况下表现出完全相同的严格的收敛速度。我们的理论保证是通过对GD轨迹进行严格分析而得出的，特别是对于模型的不同激活模式。具体而言，我们开发了对系统动力学的有效控制以限制决策边界的轨迹变化，克服了由架构非线性本质引入的主要技术挑战。 这段文字描述了一个关于梯度下降在特定神经网络结构上收敛速率的研究结果，并强调了该研究的重要性和新发现的理论基础。
URL
https://arxiv.org/abs/2603.02095
PDF
-
Learning to Attack: A Bandit Approach to Adversarial Context Poisoning
2026-02-28 09:34:06Ray Telikani, Amir H. GandomiAbstract
Neural contextual bandits are vulnerable to adversarial attacks, where subtle perturbations to rewards, actions, or contexts induce suboptimal decisions. We introduce AdvBandit, a black-box adaptive attack that formulates context poisoning as a continuous-armed bandit problem, enabling the attacker to jointly learn and exploit the victim's evolving policy. The attacker requires no access to the victim's internal parameters, reward function, or gradient information; instead, it constructs a surrogate model using a maximum-entropy inverse reinforcement learning module from observed context-action pairs and optimizes perturbations against this surrogate using projected gradient descent. An upper confidence bound-aware Gaussian process guides arm selection. An attack-budget control mechanism is also introduced to limit detection risk and overhead. We provide theoretical guarantees, including sublinear attacker regret and lower bounds on victim regret linear in the number of attacks. Experiments on three real-world datasets (Yelp, MovieLens, and Disin) against various victim contextual bandits demonstrate that our attack model achieves higher cumulative victim regret than state-of-the-art baselines.
Abstract (translated)
神经上下文型多臂赌博机容易受到对抗性攻击的影响，这种攻击通过对奖励、动作或背景信息进行细微的扰动来诱导次优决策。我们引入了AdvBandit这一黑盒自适应攻击方法，它将上下文污染问题作为连续行动的多臂赌博机问题进行建模，并使攻击者能够同时学习和利用受害者的不断变化的策略。这种攻击无需访问受害者内部参数、奖励函数或梯度信息；相反，通过使用从观察到的背景动作对中构建的最大熵逆强化学习模块，来创建一个替代模型，并通过对这个替代模型使用投影梯度下降法进行优化以生成扰动。臂选择由上置信边界感知高斯过程指导。还引入了攻击预算控制机制来限制检测风险和开销。我们提供了理论保证，包括攻击者的次线性遗憾以及与攻击数量成正比的受害者线性遗憾下限。在三个真实世界数据集（Yelp、MovieLens 和 Disin）上针对各种受害者的上下文型多臂赌博机进行实验后发现，我们的攻击模型实现了更高的累积受害者遗憾值，超过了最先进的基准方法。
URL
https://arxiv.org/abs/2603.00567
PDF
-
Geodesic Gradient Descent: A Generic and Learning-rate-free Optimizer on Objective Function-induced Manifolds
2026-02-28 02:04:15Liwei Hu, Guangyao Li, Wenyong Wang, Xiaoming Zhang, Yu XiangAbstract
Euclidean gradient descent algorithms barely capture the geometry of objective function-induced hypersurfaces and risk driving update trajectories off the hypersurfaces. Riemannian gradient descent algorithms address these issues but fail to represent complex hypersurfaces via a single classic manifold. We propose geodesic gradient descent (GGD), a generic and learning-rate-free Riemannian gradient descent algorithm. At each iteration, GGD uses an n-dimensional sphere to approximate a local neighborhood on the objective function-induced hypersurface, adapting to arbitrarily complex geometries. A tangent vector derived from the Euclidean gradient is projected onto the sphere to form a geodesic, ensuring the update trajectory stays on the hypersurface. Parameter updates are performed using the endpoint of the geodesic. The maximum step size of the gradient in GGD is equal to a quarter of the arc length on the n-dimensional sphere, thus eliminating the need for a learning rate. Experimental results show that compared with the classic Adam algorithm, GGD achieves test MSE reductions ranging from 35.79% to 48.76% for fully connected networks on the Burgers' dataset, and cross-entropy loss reductions ranging from 3.14% to 11.59% for convolutional neural networks on the MNIST dataset.
Abstract (translated)
欧几里得梯度下降算法几乎无法捕捉目标函数生成的超曲面几何结构，并且可能使更新轨迹偏离该超曲面。黎曼梯度下降算法解决了这些问题，但未能通过单一的经典流形表示复杂的超曲面。我们提出了一种通用且无需学习率的黎曼梯度下降算法——测地线梯度下降（Geodesic Gradient Descent, GGD）。在每次迭代中，GGD 使用一个 n 维球体来近似目标函数生成的超曲面上的一个局部邻域，并适应任意复杂的几何结构。通过将由欧几里得梯度导出的切向量投影到该球体上形成测地线，确保更新轨迹保持在超曲面上。参数更新使用的是测地线的端点进行。GGD 中梯度的最大步长等于 n 维球上的四分之一弧长，从而消除了对学习率的需求。 实验结果表明，与经典的 Adam 算法相比，在 Burgers 数据集上用于完全连接网络时，GGD 可实现测试 MSE 的减少范围为 35.79% 至 48.76%，在 MNIST 数据集上的卷积神经网络则可实现交叉熵损失的减少范围为 3.14% 至 11.59%。
URL
https://arxiv.org/abs/2603.06651
PDF
-
DP-aware AdaLN-Zero: Taming Conditioning-Induced Heavy-Tailed Gradients in Differentially Private Diffusion
2026-02-26 04:32:07Tao Huang, Jiayang Meng, Xu Yang, Chen Hou, Hong ChenAbstract
Condition injection enables diffusion models to generate context-aware outputs, which is essential for many time-series tasks. However, heterogeneous conditional contexts (e.g., observed history, missingness patterns or outlier covariates) can induce heavy-tailed per-example gradients. Under Differentially Private Stochastic Gradient Descent (DP-SGD), these rare conditioning-driven heavy-tailed gradients disproportionately trigger global clipping, resulting in outlier-dominated updates, larger clipping bias, and degraded utility under a fixed privacy budget. In this paper, we propose DP-aware AdaLN-Zero, a drop-in sensitivity-aware conditioning mechanism for conditional diffusion transformers that limits conditioning-induced gain without modifying the DP-SGD mechanism. DP-aware AdaLN-Zero jointly constrains conditioning representation magnitude and AdaLN modulation parameters via bounded re-parameterization, suppressing extreme gradient tail events before gradient clipping and noise injection. Empirically, DP-SGD equipped with DP-aware AdaLN-Zero improves interpolation/imputation and forecasting under matched privacy settings. We observe consistent gains on a real-world power dataset and two public ETT benchmarks over vanilla DP-SGD. Moreover, gradient diagnostics attribute these improvements to conditioning-specific tail reshaping and reduced clipping distortion, while preserving expressiveness in non-private training. Overall, these results show that sensitivity-aware conditioning can substantially improve private conditional diffusion training without sacrificing standard performance.
Abstract (translated)
条件注入使得扩散模型能够生成上下文感知的输出，这对于许多时间序列任务至关重要。然而，异构条件上下文（例如观察历史、缺失模式或异常协变量）可能会导致长尾分布的梯度。在差异隐私随机梯度下降法(DP-SGD)中，这些罕见的由条件驱动的重尾梯度会不恰当地触发全局剪切操作，从而导致异常值主导的更新过程，并产生更大的剪切偏差和固定的隐私预算下的功能降级。 本文提出了一种新的方法DP-aware AdaLN-Zero，这是一种敏感性感知条件注入机制，适用于有条件扩散变换器，在无需修改DP-SGD机制的情况下限制了由条件引入的变化。DP-aware AdaLN-Zero通过有界重新参数化同时约束条件表示的幅度和AdaLN调制参数，抑制极端梯度尾部事件，并在梯度剪切和噪声添加之前消除这些异常值。 实验表明，配备了DP-aware AdaLN-Zero的DP-SGD在匹配隐私设置下提高了插值/填补和预测性能。我们在一个真实世界电力数据集以及两个公开ETT基准测试中观察到了相对于未改进的DP-SGD的一致性收益。此外，梯度诊断将这些改进归因于条件特定尾部重塑和剪切失真的减少，并在非隐私训练过程中保持了表达能力。 总体而言，这些结果表明敏感性感知条件注入可以在不牺牲标准性能的情况下显著提高私人有条件扩散训练的效果。
URL
https://arxiv.org/abs/2602.22610
PDF
