Gradient_Descent
-
Generalisation of RLHF under Reward Shift and Clipped KL Regularisation
2026-02-25 10:36:17Kenton Tang, Yuzhu Chen, Fengxiang HeAbstract
Alignment and adaptation in large language models heavily rely on reinforcement learning from human feedback (RLHF); yet, theoretical understanding of its generalisability remains premature, especially when the learned reward could shift, and the KL control is estimated and clipped. To address this issue, we develop generalisation theory for RLHF that explicitly accounts for (1) \emph{reward shift}: reward models are trained on preference data from earlier or mixed behaviour policies while RLHF optimises the current policy on its own rollouts; and (2) \emph{clipped KL regularisation}: the KL regulariser is estimated from sampled log-probability ratios and then clipped for stabilisation, resulting in an error to RLHF. We present generalisation bounds for RLHF, suggesting that the generalisation error stems from a sampling error from prompts and rollouts, a reward shift error, and a KL clipping error. We also discuss special cases of (1) initialising RLHF parameters with a uniform prior over a finite space, and (2) training RLHF by stochastic gradient descent, as an Ornstein-Uhlenbeck process. The theory yields practical implications in (1) optimal KL clipping threshold, and (2) budget allocation in prompts, rollouts, and preference data.
Abstract (translated)
大型语言模型的对齐和适应很大程度上依赖于来自人类反馈的强化学习（RLHF），然而，关于其泛化能力的理论理解仍处于初级阶段，尤其是在学到的奖励可能发生变化且KL控制被估计并裁剪的情况下。为了解决这一问题，我们开发了针对RLHF的泛化理论，该理论明确考虑了以下两点： 1. **奖励漂移（Reward Shift）**：奖励模型是在早期或混合行为策略的数据上训练的，而RLHF则是基于当前策略在自身生成的rollout数据进行优化。 2. **裁剪KL正则化（Clipped KL Regularisation）**：KL正则项是从抽样对数概率比估计得到，并随后被裁剪以稳定算法，这会导致RLHF中的误差。 我们提出了针对RLHF的泛化边界，表明其泛化错误源自于从提示和rollout抽样的样本误差、奖励漂移引起的误差以及KL裁剪导致的误差。此外，还讨论了两个特殊情形： 1. RLHF参数初始化为有限空间上的均匀先验。 2. 通过随机梯度下降（SGD）训练RLHF，视其为奥恩斯坦-乌伦贝克过程。 该理论在实践中具有重要的意义，包括确定最佳的KL裁剪阈值以及如何分配提示、rollout和偏好数据的成本预算。
URL
https://arxiv.org/abs/2602.21765
PDF
-
AHC: Meta-Learned Adaptive Compression for Continual Object Detection on Memory-Constrained Microcontrollers
2026-02-24 07:06:51Bibin WilsonAbstract
Deploying continual object detection on microcontrollers (MCUs) with under 100KB memory requires efficient feature compression that can adapt to evolving task distributions. Existing approaches rely on fixed compression strategies (e.g., FiLM conditioning) that cannot adapt to heterogeneous task characteristics, leading to suboptimal memory utilization and catastrophic forgetting. We introduce Adaptive Hierarchical Compression (AHC), a meta-learning framework featuring three key innovations: (1) true MAML-based compression that adapts via gradient descent to each new task in just 5 inner-loop steps, (2) hierarchical multi-scale compression with scale-aware ratios (8:1 for P3, 6.4:1 for P4, 4:1 for P5) matching FPN redundancy patterns, and (3) a dual-memory architecture combining short-term and long-term banks with importance-based consolidation under a hard 100KB budget. We provide formal theoretical guarantees bounding catastrophic forgetting as O({\epsilon}{this http URL(T)} + 1/{this http URL(M)}) where {\epsilon} is compression error, T is task count, and M is memory size. Experiments on CORe50, TiROD, and PASCAL VOC benchmarks with three standard baselines (Fine-tuning,EWC, iCaRL) demonstrate that AHC enables practical continual detection within a 100KB replay budget, achieving competitive accuracy through mean-pooled compressed feature replay combined with EWC regularization and feature distillation.
Abstract (translated)
URL
https://arxiv.org/abs/2604.09576
PDF
-
Nacrith: Neural Lossless Compression via Ensemble Context Modeling and High-Precision CDF Coding
2026-02-23 09:14:05Roberto TacconelliAbstract
We present Nacrith, a lossless compression system that combines a 135M-parameter transformer language model (SmolLM2-135M) with an ensemble of lightweight online predictors and a 32-bit arithmetic coder. Beyond the base LLM-plus-arithmetic-coding paradigm, Nacrith introduces several contributions: (1) a CDF precision upgrade from 2^16 to 2^24 that eliminates ~75% of quantization overhead caused by minimum-probability floors in large vocabularies; (2) a token-level N-gram model for fast local predictions; (3) an adaptive log-space bias head correcting per-document LLM errors via online gradient descent; (4) confidence-based LLM skip for accelerating highly predictable tokens; (5) a hybrid binary format (NC06) extending neural compression to arbitrary binary files--to our knowledge a first among LLM-based compressors; (6) a this http URL inference backend achieving ~7x faster single-token decode than PyTorch; (7) parallel multi-GPU compression across up to 8 workers; and (8) native KV cache sliding window reducing per-slide cost by ~37x. The system requires only ~500 MB of GGUF weights and ~1.2 GB VRAM per worker, running on consumer GPUs. On this http URL (Canterbury Corpus, 152 KB), Nacrith achieves 0.918 bits per byte (bpb)--outperforming gzip by 3.1x, bzip2 by 2.5x, CMIX v21 by 44%, and ts_zip by 20%, while compressing below the 0th-, 1st-, and 2nd-order byte-level Shannon entropy bounds. On enwik8 (100 MB), Nacrith achieves 0.9389 bpb (11.74%), surpassing ts_zip (~1.11 bpb) by 15% and FineZip (1.024 bpb) by 8% despite using a 60x smaller model with no fine-tuning. An out-of-distribution evaluation on a document published after the model's training cutoff confirms these gains are not memorization artifacts, achieving 0.723 bpb on unseen text.
Abstract (translated)
我们介绍了Nacrith，这是一个结合了1.35亿参数的变压器语言模型（SmolLM2-135M）、一组轻量级在线预测器以及一个32位算术编码器的无损压缩系统。除了基础的大规模语言模型（LLM）加算术编码的基本框架之外，Nacrith还引入了以下几项创新： 1. 将CDF精度从\(2^{16}\)提升到\(2^{24}\)，消除了大约75%由于大型词汇表中最小概率地板导致的量化开销。 2. 一种用于快速局部预测的令牌级N-gram模型。 3. 一个自适应对数空间偏置头，通过在线梯度下降纠正每份文档中的LLM错误。 4. 基于置信度的LLM跳过机制，以加速高度可预测的令牌。 5. 混合二进制格式（NC06），将神经压缩扩展到任意二进制文件中——据我们所知，在基于大模型的压缩器中是首创。 6. 一个推理后端，实现了比PyTorch快约7倍的单令牌解码速度。 7. 在多达8个工作线程上进行跨多个GPU的并行压缩。 8. 原生KV缓存滑动窗口，将每次滑动的成本降低了大约37倍。 该系统仅需约500 MB GGUF权重和每个工作线程1.2 GB VRAM，在消费级GPU上运行。在Canterbury语料库（152 KB）测试中，Nacrith实现了每字节0.918比特的压缩率——比gzip高出3.1倍、bzip2高出2.5倍、CMIX v21高出44%，以及ts_zip高出20%。同时，它还低于零次、一次和二次字级香农熵限值。 在enwik8（100 MB）测试中，Nacrith实现了每字节0.9389比特的压缩率——比ts_zip（约1.11比特/字节）高出15%，并比FineZip（1.024比特/字节）高出8%。值得注意的是，它使用了一个比后者小60倍且未进行微调的模型。 一项超出训练截止日期后发布文档的数据分布评估证实了这些性能提升并非记忆效应的结果，在未知文本上的压缩率为每字节0.723比特。
URL
https://arxiv.org/abs/2602.19626
PDF
-
Deep LoRA-Unfolding Networks for Image Restoration
2026-02-21 02:57:48Xiangming Wang, Haijin Zeng, Benteng Sun, Jiezhang Cao, Kai Zhang, Qiangqiang Shen, Yongyong ChenAbstract
Deep unfolding networks (DUNs), combining conventional iterative optimization algorithms and deep neural networks into a multi-stage framework, have achieved remarkable accomplishments in Image Restoration (IR), such as spectral imaging reconstruction, compressive sensing and this http URL unfolds the iterative optimization steps into a stack of sequentially linked this http URL block consists of a Gradient Descent Module (GDM) and a Proximal Mapping Module (PMM) which is equivalent to a denoiser from a Bayesian perspective, operating on Gaussian noise with a known this http URL, existing DUNs suffer from two critical limitations: (i) their PMMs share identical architectures and denoising objectives across stages, ignoring the need for stage-specific adaptation to varying noise levels; and (ii) their chain of structurally repetitive blocks results in severe parameter redundancy and high memory consumption, hindering deployment in large-scale or resource-constrained this http URL address these challenges, we introduce generalized Deep Low-rank Adaptation (LoRA) Unfolding Networks for image restoration, named LoRun, harmonizing denoising objectives and adapting different denoising levels between stages with compressed memory usage for more efficient this http URL introduces a novel paradigm where a single pretrained base denoiser is shared across all stages, while lightweight, stage-specific LoRA adapters are injected into the PMMs to dynamically modulate denoising behavior according to the noise level at each unfolding this http URL design decouples the core restoration capability from task-specific adaptation, enabling precise control over denoising intensity without duplicating full network parameters and achieving up to $N$ times parameter reduction for an $N$-stage DUN with on-par or better this http URL experiments conducted on three IR tasks validate the efficiency of our method.
Abstract (translated)
结合传统迭代优化算法和深度神经网络的深层展开网络（DUNs）在图像恢复领域，如光谱成像重建、压缩感知等任务中取得了显著成就。这类方法通过将迭代步骤拆解为一系列顺序连接的模块化块来工作，每个块包含一个梯度下降模块（GDM）和一个临近映射模块（PMM），后者从贝叶斯视角来看相当于一种去噪器，针对已知高斯噪声进行操作。然而，现有的DUNs面临两个关键限制：(i)它们的PMM在各阶段共享相同的架构及去噪目标，忽视了根据不同阶段所需的特定适应性调整；(ii)重复结构导致严重的参数冗余和高内存消耗，阻碍其在大规模或资源受限环境中的部署。 为解决这些问题，我们提出了广义深度低秩适配（LoRA）展开网络用于图像恢复，命名为LoRun。该方法通过压缩的内存使用量实现了不同阶段去噪目标之间的协调及适应不同的噪声水平，并且提高了效率。这种方法引入了一种新颖的方法：在整个过程中共享一个预训练的基本去噪器，同时在每个PMM中插入轻量级、阶段特定的LoRA适配器来根据展开步骤中的噪声级别动态调节去噪行为。 该设计解耦了核心恢复能力与任务特定适应性调整，从而能够在不复制完整网络参数的情况下实现对去噪强度的精确控制，并且最多可以将N个阶段DUN的参数减少到原来的1/N倍，在保持性能不变或更好的情况下。通过在三个图像恢复任务上的实验验证了我们方法的有效性。
URL
https://arxiv.org/abs/2602.18697
PDF
-
BloomNet: Exploring Single vs. Multiple Object Annotation for Flower Recognition Using YOLO Variants
2026-02-20 19:47:45Safwat Nusrat, Prithwiraj BhattacharjeeAbstract
Precise localization and recognition of flowers are crucial for advancing automated agriculture, particularly in plant phenotyping, crop estimation, and yield monitoring. This paper benchmarks several YOLO architectures such as YOLOv5s, YOLOv8n/s/m, and YOLOv12n for flower object detection under two annotation regimes: single-image single-bounding box (SISBB) and single-image multiple-bounding box (SIMBB). The FloralSix dataset, comprising 2,816 high-resolution photos of six different flower species, is also introduced. It is annotated for both dense (clustered) and sparse (isolated) scenarios. The models were evaluated using Precision, Recall, and Mean Average Precision (mAP) at IoU thresholds of 0.5 (mAP@0.5) and 0.5-0.95 (mAP@0.5:0.95). In SISBB, YOLOv8m (SGD) achieved the best results with Precision 0.956, Recall 0.951, mAP@0.5 0.978, and mAP@0.5:0.95 0.865, illustrating strong accuracy in detecting isolated flowers. With mAP@0.5 0.934 and mAP@0.5:0.95 0.752, YOLOv12n (SGD) outperformed the more complicated SIMBB scenario, proving robustness in dense, multi-object detection. Results show how annotation density, IoU thresholds, and model size interact: recall-optimized models perform better in crowded environments, whereas precision-oriented models perform best in sparse scenarios. In both cases, the Stochastic Gradient Descent (SGD) optimizer consistently performed better than alternatives. These density-sensitive sensors are helpful for non-destructive crop analysis, growth tracking, robotic pollination, and stress evaluation.
Abstract (translated)
精确的花朵定位和识别对于推动自动化农业至关重要，尤其是在植物表型分析、作物估算以及产量监测方面。本文评估了多个YOLO架构（如YOLOv5s、YOLOv8n/s/m 和 YOLOv12n），它们在两种标注体系下用于花物体检测：单图单一边界框（SISBB）和单图多边界框（SIMBB）。文中还介绍了FloralSix数据集，该数据集包含6种不同花卉物种的2,816张高分辨率照片，并且这些图片既标有密集（簇状）场景也标有稀疏（孤立）场景。使用精度、召回率以及在IoU阈值为0.5 (mAP@0.5) 和 0.5-0.95 (mAP@0.5:0.95) 下的平均精度均值(mAP)对模型进行评估。 在SISBB中，YOLOv8m（SGD）取得了最佳结果：精确度为0.956，召回率为0.951，mAP@0.5为0.978，mAP@0.5:0.95为0.865。这表明在检测孤立花朵时具有很强的准确性。而在SIMBB中，YOLOv12n（SGD）以mAP@0.5 0.934 和 mAP@0.5:0.95 0.752 的成绩超过了更为复杂的场景，在密集、多目标检测方面表现出了强大的鲁棒性。结果表明了标注密度、IoU阈值和模型大小之间的相互作用：优化召回率的模型在拥挤环境中表现出色，而侧重于精度的模型则在稀疏场景中表现最佳。两种情况下，随机梯度下降（SGD）优化器的表现均优于其他选择。 这些灵敏度不同的传感器对于非破坏性作物分析、生长监测、机器人授粉以及压力评估等非常有用。
URL
https://arxiv.org/abs/2602.18585
PDF
-
Bayesian Optimality of In-Context Learning with Selective State Spaces
2026-02-19 12:41:28Di Zhang, Jiaqi XingAbstract
We propose Bayesian optimal sequential prediction as a new principle for understanding in-context learning (ICL). Unlike interpretations framing Transformers as performing implicit gradient descent, we formalize ICL as meta-learning over latent sequence tasks. For tasks governed by Linear Gaussian State Space Models (LG-SSMs), we prove a meta-trained selective SSM asymptotically implements the Bayes-optimal predictor, converging to the posterior predictive mean. We further establish a statistical separation from gradient descent, constructing tasks with temporally correlated noise where the optimal Bayesian predictor strictly outperforms any empirical risk minimization (ERM) estimator. Since Transformers can be seen as performing implicit ERM, this demonstrates selective SSMs achieve lower asymptotic risk due to superior statistical efficiency. Experiments on synthetic LG-SSM tasks and a character-level Markov benchmark confirm selective SSMs converge faster to Bayes-optimal risk, show superior sample efficiency with longer contexts in structured-noise settings, and track latent states more robustly than linear Transformers. This reframes ICL from "implicit optimization" to "optimal inference," explaining the efficiency of selective SSMs and offering a principled basis for architecture design.
Abstract (translated)
我们提出了贝叶斯最优顺序预测作为理解上下文学习（ICL）的新原则。与将变压器解释为执行隐式梯度下降的框架不同，我们将ICL形式化为对潜在序列任务进行元学习。对于由线性高斯状态空间模型（LG-SSM）支配的任务，我们证明了经过元训练的选择性SSM在渐近意义上实现了贝叶斯最优预测器，并且收敛于后验预测均值。此外，我们在构建具有时间相关噪声的任务时建立了与梯度下降的统计分离，使得最佳贝叶斯预测器严格优于任何经验风险最小化（ERM）估计器。由于变压器可以视为执行隐式ERM，这表明选择性SSM通过更优的统计效率实现了更低渐近风险。在合成LG-SSM任务和字符级马尔可夫基准上的实验证实了选择性SSM更快地收敛于贝叶斯最优风险，在结构噪声环境中具有更好的样本效率，并且比线性变压器更稳健地跟踪潜在状态。这将ICL从“隐式优化”重新框架为“最佳推理”，解释了选择性SSM的高效性和架构设计的原则基础。
URL
https://arxiv.org/abs/2602.17744
PDF
-
Conjugate Learning Theory: Uncovering the Mechanisms of Trainability and Generalization in Deep Neural Networks
2026-02-18 04:26:55Binchuan QiAbstract
In this work, we propose a notion of practical learnability grounded in finite sample settings, and develop a conjugate learning theoretical framework based on convex conjugate duality to characterize this learnability property. Building on this foundation, we demonstrate that training deep neural networks (DNNs) with mini-batch stochastic gradient descent (SGD) achieves global optima of empirical risk by jointly controlling the extreme eigenvalues of a structure matrix and the gradient energy, and we establish a corresponding convergence theorem. We further elucidate the impact of batch size and model architecture (including depth, parameter count, sparsity, skip connections, and other characteristics) on non-convex optimization. Additionally, we derive a model-agnostic lower bound for the achievable empirical risk, theoretically demonstrating that data determines the fundamental limit of trainability. On the generalization front, we derive deterministic and probabilistic bounds on generalization error based on generalized conditional entropy measures. The former explicitly delineates the range of generalization error, while the latter characterizes the distribution of generalization error relative to the deterministic bounds under independent and identically distributed (i.i.d.) sampling conditions. Furthermore, these bounds explicitly quantify the influence of three key factors: (i) information loss induced by irreversibility in the model, (ii) the maximum attainable loss value, and (iii) the generalized conditional entropy of features with respect to labels. Moreover, they offer a unified theoretical lens for understanding the roles of regularization, irreversible transformations, and network depth in shaping the generalization behavior of deep neural networks. Extensive experiments validate all theoretical predictions, confirming the framework's correctness and consistency.
Abstract (translated)
URL
https://arxiv.org/abs/2602.16177
PDF
-
The Geometry of Alignment Collapse: When Fine-Tuning Breaks Safety
2026-02-17 18:39:15Max Springer, Chung Peng Lee, Blossom Metevier, Jane Castleman, Bohdan Turbal, Hayoung Jung, Zeyu Shen, Aleksandra KorolovaAbstract
Fine-tuning aligned language models on benign tasks unpredictably degrades safety guardrails, even when training data contains no harmful content and developers have no adversarial intent. We show that the prevailing explanation, that fine-tuning updates should be orthogonal to safety-critical directions in high-dimensional parameter space, offers false reassurance: we show this orthogonality is structurally unstable and collapses under the dynamics of gradient descent. We then resolve this through a novel geometric analysis, proving that alignment concentrates in low-dimensional subspaces with sharp curvature, creating a brittle structure that first-order methods cannot detect or defend. While initial fine-tuning updates may indeed avoid these subspaces, the curvature of the fine-tuning loss generates second-order acceleration that systematically steers trajectories into alignment-sensitive regions. We formalize this mechanism through the Alignment Instability Condition, three geometric properties that, when jointly satisfied, lead to safety degradation. Our main result establishes a quartic scaling law: alignment loss grows with the fourth power of training time, governed by the sharpness of alignment geometry and the strength of curvature coupling between the fine-tuning task and safety-critical parameters. These results expose a structural blind spot in the current safety paradigm. The dominant approaches to safe fine-tuning address only the initial snapshot of a fundamentally dynamic problem. Alignment fragility is not a bug to be patched; it is an intrinsic geometric property of gradient descent on curved manifolds. Our results motivate the development of curvature-aware methods, and we hope will further enable a shift in alignment safety analysis from reactive red-teaming to predictive diagnostics for open-weight model deployment.
Abstract (translated)
URL
https://arxiv.org/abs/2602.15799
PDF
-
Decision Making under Imperfect Recall: Algorithms and Benchmarks
2026-02-16 23:19:01Emanuel Tewolde, Brian Hu Zhang, Ioannis Anagnostides, Tuomas Sandholm, Vincent ConitzerAbstract
In game theory, imperfect-recall decision problems model situations in which an agent forgets information it held before. They encompass games such as the ``absentminded driver'' and team games with limited communication. In this paper, we introduce the first benchmark suite for imperfect-recall decision problems. Our benchmarks capture a variety of problem types, including ones concerning privacy in AI systems that elicit sensitive information, and AI safety via testing of agents in simulation. Across 61 problem instances generated using this suite, we evaluate the performance of different algorithms for finding first-order optimal strategies in such problems. In particular, we introduce the family of regret matching (RM) algorithms for nonlinear constrained optimization. This class of parameter-free algorithms has enjoyed tremendous success in solving large two-player zero-sum games, but, surprisingly, they were hitherto relatively unexplored beyond that setting. Our key finding is that RM algorithms consistently outperform commonly employed first-order optimizers such as projected gradient descent, often by orders of magnitude. This establishes, for the first time, the RM family as a formidable approach to large-scale constrained optimization problems.
Abstract (translated)
URL
https://arxiv.org/abs/2602.15252
PDF
-
RPGD: RANSAC-P3P Gradient Descent for Extrinsic Calibration in 3D Human Pose Estimation
Abstract
In this paper, we propose RPGD (RANSAC-P3P Gradient Descent), a human-pose-driven extrinsic calibration framework that robustly aligns MoCap-based 3D skeletal data with monocular or multi-view RGB cameras using only natural human motion. RPGD formulates extrinsic calibration as a coarse-to-fine problem tailored to human poses, combining the global robustness of RANSAC-P3P with Gradient-Descent-based refinement. We evaluate RPGD on three large-scale public 3D HPE datasets as well as on a self-collected in-the-wild dataset. Experimental results demonstrate that RPGD consistently recovers extrinsic parameters with accuracy comparable to the provided ground truth, achieving sub-pixel MPJPE reprojection error even in challenging, noisy settings. These results indicate that RPGD provides a practical and automatic solution for reliable extrinsic calibration of large-scale 3D HPE dataset collection.
Abstract (translated)
URL
https://arxiv.org/abs/2602.13901
PDF
-
A Wavefield Correlation Approach to Improve Sound Speed Estimation in Ultrasound Autofocusing
2026-02-13 10:40:20Louise Zhuang, Samuel Beuret, Ben Frey, Saachi Munot, Jeremy J. DahlAbstract
Aberration often degrades ultrasound image quality when beamforming does not account for wavefront distortions. In the past decade, local sound speed estimators have been developed for distributed aberration correction throughout a medium. Recently, iterative sound speed optimization approaches have achieved more accurate estimates than earlier approaches, but these newer methods still struggle with decreased accuracy for media with reverberation clutter and large sound speed changes. To address these challenges, we propose using a wavefield correlation (WFC) beamforming approach when performing sound speed optimization. WFC correlates simulated forward-propagated transmit wavefields and backwards-propagated receive wavefields in order to form images. This process more accurately models wave propagation in heterogeneous media and can decrease diffuse clutter due to its spatiotemporal matched filtering effect. This beamformer is implemented using auto-differentiation software to then perform gradient descent optimization, using a total-variation regularized common midpoint phase focus metric loss, on the local sound speed map used during beamforming. This approach is compared to using delay and sum (DAS) with straight-ray time delay calculations in the same sound speed optimization approach on a variety of simulated, phantom, and in vivo data with large sound speed changes and clutter. Results show that using WFC decreases sound speed estimation error, and using the estimates for aberration correction improves image resolution and contrast. These promising results have potential to improve pulse-echo imaging for challenging clinical scenarios.
Abstract (translated)
URL
https://arxiv.org/abs/2602.12805
PDF
-
Composable Model-Free RL for Navigation with Input-Affine Systems
2026-02-13 00:19:35Xinhuan Sang, Abdelrahman Abdelgawad, Roberto TronAbstract
As autonomous robots move into complex, dynamic real-world environments, they must learn to navigate safely in real time, yet anticipating all possible behaviors is infeasible. We propose a composable, model-free reinforcement learning method that learns a value function and an optimal policy for each individual environment element (e.g., goal or obstacle) and composes them online to achieve goal reaching and collision avoidance. Assuming unknown nonlinear dynamics that evolve in continuous time and are input-affine, we derive a continuous-time Hamilton-Jacobi-Bellman (HJB) equation for the value function and show that the corresponding advantage function is quadratic in the action and optimal policy. Based on this structure, we introduce a model-free actor-critic algorithm that learns policies and value functions for static or moving obstacles using gradient descent. We then compose multiple reach/avoid models via a quadratically constrained quadratic program (QCQP), yielding formal obstacle-avoidance guarantees in terms of value-function level sets, providing a model-free alternative to CLF/CBF-based controllers. Simulations demonstrate improved performance over a PPO baseline applied to a discrete-time approximation.
Abstract (translated)
URL
https://arxiv.org/abs/2602.12492
PDF
-
On the implicit regularization of Langevin dynamics with projected noise
2026-02-12 18:45:42Govind Menon, Austin J. Stromme, Adrien VacherAbstract
We study Langevin dynamics with noise projected onto the directions orthogonal to an isometric group action. This mathematical model is introduced to shed new light on the effects of symmetry on stochastic gradient descent for over-parametrized models. Our main result identifies a novel form of implicit regularization: when the initial and target density are both invariant under the group action, Langevin dynamics with projected noise is equivalent in law to Langevin dynamics with isotropic diffusion but with an additional drift term proportional to the negative log volume of the group orbit. We prove this result by constructing a coupling of the two processes via a third process on the group itself, and identify the additional drift as the mean curvature of the orbits.
Abstract (translated)
我们研究了噪声投影到等距群作用正交方向上的兰格vin动力学。这一数学模型被引入以揭示对称性在过参数化模型的随机梯度下降中的影响。我们的主要结果确定了一种新型的隐式正则化形式：当初始密度和目标密度都在群作用下不变时，带有投影噪声的兰格vin动力学与具有额外漂移项（该漂移项与群轨道的负对数体积成比例）的各向同性扩散下的兰格vin动力学在法理上是等价的。我们通过构造两个过程之间的耦合（即在群自身上的第三个过程中），证明了这一结果，并将额外的漂移项识别为轨道的平均曲率。
URL
https://arxiv.org/abs/2602.12257
PDF
-
Gradient Compression May Hurt Generalization: A Remedy by Synthetic Data Guided Sharpness Aware Minimization
2026-02-12 05:08:49Yujie Gu, Richeng Jin, Zhaoyang Zhang, Huaiyu DaiAbstract
It is commonly believed that gradient compression in federated learning (FL) enjoys significant improvement in communication efficiency with negligible performance degradation. In this paper, we find that gradient compression induces sharper loss landscapes in federated learning, particularly under non-IID data distributions, which suggests hindered generalization capability. The recently emerging Sharpness Aware Minimization (SAM) effectively searches for a flat minima by incorporating a gradient ascent step (i.e., perturbing the model with gradients) before the celebrated stochastic gradient descent. Nonetheless, the direct application of SAM in FL suffers from inaccurate estimation of the global perturbation due to data heterogeneity. Existing approaches propose to utilize the model update from the previous communication round as a rough estimate. However, its effectiveness is hindered when model update compression is incorporated. In this paper, we propose FedSynSAM, which leverages the global model trajectory to construct synthetic data and facilitates an accurate estimation of the global perturbation. The convergence of the proposed algorithm is established, and extensive experiments are conducted to validate its effectiveness.
Abstract (translated)
人们普遍认为，在联邦学习（FL）中，梯度压缩可以显著提高通信效率，并且几乎不会导致性能下降。然而，在本文中，我们发现梯度压缩在联邦学习中会导致更尖锐的损失景观，尤其是在非独立同分布（non-IID）的数据分布下，这表明泛化能力受到了阻碍。 最近出现的敏锐度感知最小化（SAM）通过在传统的随机梯度下降之前增加一个梯度上升步骤（即用梯度扰动模型），有效地寻找平坦的极小值。然而，在联邦学习中直接应用SAM会因为数据异质性而导致全局扰动估计不准确的问题。现有方法建议使用上一轮通信中的模型更新作为粗略估计，但当引入模型更新压缩时，这种方法的有效性就会受到限制。 在本文中，我们提出了一种名为FedSynSAM的方法，该方法利用全球模型轨迹构建合成数据，并促进对全局扰动的精确估计。我们建立了所提算法的收敛性理论，并通过广泛的实验验证了其有效性。
URL
https://arxiv.org/abs/2602.11584
PDF
-
SCRAPL: Scattering Transform with Random Paths for Machine Learning
2026-02-11 18:57:08Christopher Mitcheltree, Vincent Lostanlen, Emmanouil Benetos, Mathieu LagrangeAbstract
The Euclidean distance between wavelet scattering transform coefficients (known as paths) provides informative gradients for perceptual quality assessment of deep inverse problems in computer vision, speech, and audio processing. However, these transforms are computationally expensive when employed as differentiable loss functions for stochastic gradient descent due to their numerous paths, which significantly limits their use in neural network training. Against this problem, we propose "Scattering transform with Random Paths for machine Learning" (SCRAPL): a stochastic optimization scheme for efficient evaluation of multivariable scattering transforms. We implement SCRAPL for the joint time-frequency scattering transform (JTFS) which demodulates spectrotemporal patterns at multiple scales and rates, allowing a fine characterization of intermittent auditory textures. We apply SCRAPL to differentiable digital signal processing (DDSP), specifically, unsupervised sound matching of a granular synthesizer and the Roland TR-808 drum machine. We also propose an initialization heuristic based on importance sampling, which adapts SCRAPL to the perceptual content of the dataset, improving neural network convergence and evaluation performance. We make our code and audio samples available and provide SCRAPL as a Python package.
Abstract (translated)
波lets变换系数（也称为路径）之间的欧几里得距离为计算机视觉、语音和音频处理中的深度逆问题的感知质量评估提供了有信息量的梯度。然而，由于这些变换具有许多路径，在用作随机梯度下降（SGD）的可微损失函数时计算成本高昂，这极大地限制了它们在神经网络训练中的应用。为了解决这个问题，我们提出了“机器学习中带有随机路径的散射变换”(SCRAPL)：一种高效的多变量散射变换评估的随机优化方案。我们在联合时频散射变换(JTFS)上实现了SCRAPL，该变换可以在多个尺度和速率下解调频谱-时间模式，并允许对间歇性听觉纹理进行精细表征。我们将SCRAPL应用于可微数字信号处理（DDSP），特别是用于颗粒合成器与Roland TR-808鼓机之间的无监督声音匹配。我们还提出了一种基于重要性采样的初始化启发式方法，该方法将SCRAPL适应于数据集的感知内容，从而提高神经网络的收敛性和评估性能。我们将代码、音频样本以及SCRAPL作为Python包提供给社区使用。
URL
https://arxiv.org/abs/2602.11145
PDF
-
Language Model Inversion through End-to-End Differentiation
2026-02-11 17:14:41Kevin Yandoka Denamgana\"i, Kartic SubrAbstract
Despite emerging research on Language Models (LM), few approaches analyse the invertibility of LMs. That is, given a LM and a desirable target output sequence of tokens, determining what input prompts would yield the target output remains an open problem. We formulate this problem as a classical gradient-based optimisation. First, we propose a simple algorithm to achieve end-to-end differentiability of a given (frozen) LM and then find optimised prompts via gradient descent. Our central insight is to view LMs as functions operating on sequences of distributions over tokens (rather than the traditional view as functions on sequences of tokens). Our experiments and ablations demonstrate that our DLM-powered inversion can reliably and efficiently optimise prompts of lengths $10$ and $80$ for targets of length $20$, for several white-box LMs (out-of-the-box).
Abstract (translated)
尽管有关语言模型（Language Models, LM）的研究不断涌现，但很少有方法分析LM的逆向可解性。也就是说，给定一个LM和期望的目标输出序列，确定哪些输入提示可以产生该目标输出仍然是一个悬而未决的问题。我们将这个问题表述为经典的基于梯度的优化问题。首先，我们提出了一种简单的算法，以实现给定的（冻结的）LM在端到端上的可微分性，然后通过梯度下降法找到最优的输入提示。我们的核心见解是将LM视为作用于令牌分布序列上的函数（而不是传统的将其视为作用于令牌序列上的函数）。实验和消融研究表明，在几个白盒语言模型中，我们基于DLM的逆向方法可以可靠且高效地优化长度为10和80的输入提示，以实现目标输出长度为20的目标。
URL
https://arxiv.org/abs/2602.11044
PDF
-
Low-Dimensional Execution Manifolds in Transformer Learning Dynamics: Evidence from Modular Arithmetic Tasks
2026-02-11 03:57:46Yongzhong XuAbstract
We investigate the geometric structure of learning dynamics in overparameterized transformer models through carefully controlled modular arithmetic tasks. Our primary finding is that despite operating in high-dimensional parameter spaces ($d=128$), transformer training trajectories rapidly collapse onto low-dimensional execution manifolds of dimension $3$--$4$. This dimensional collapse is robust across random seeds and moderate task difficulties, though the orientation of the manifold in parameter space varies between runs. We demonstrate that this geometric structure underlies several empirically observed phenomena: (1) sharp attention concentration emerges as saturation along routing coordinates within the execution manifold, (2) stochastic gradient descent (SGD) exhibits approximately integrable dynamics when projected onto the execution subspace, with non-integrability confined to orthogonal staging directions, and (3) sparse autoencoders capture auxiliary routing structure but fail to isolate execution itself, which remains distributed across the low-dimensional manifold. Our results suggest a unifying geometric framework for understanding transformer learning, where the vast majority of parameters serve to absorb optimization interference while core computation occurs in a dramatically reduced subspace. These findings have implications for interpretability, training curriculum design, and understanding the role of overparameterization in neural network learning.
Abstract (translated)
我们通过精心控制的模运算任务，研究了过度参数化变压器模型中学习动态的几何结构。我们的主要发现是：尽管在高维参数空间（$d=128$）内运行，变压器训练轨迹会迅速坍缩到维度为3至4的低维执行流形上。这种维度坍缩在随机种子和任务难度适中的情况下表现出稳健性，虽然不同实验中该流形在参数空间中的方向有所不同。 我们展示了这种几何结构是几种经验观察现象的基础：（1）注意力集中的突然增强表现为沿着执行流形内路由坐标上的饱和；（2）随机梯度下降（SGD）在投影到执行子空间时表现出近似可积的动力学特性，而非可积性仅限于正交阶段方向；（3）稀疏自动编码器捕捉到了辅助的路由结构，但未能隔离出执行过程本身，这仍分布在整个低维流形上。 我们的结果表明存在一个统一的几何框架来理解变压器学习，其中绝大多数参数用于吸收优化干扰，而核心计算则发生在大幅缩减的子空间中。这些发现对可解释性、训练课程设计以及过度参数化在神经网络学习中的作用的理解具有重要意义。
URL
https://arxiv.org/abs/2602.10496
PDF
-
Monte Carlo Maximum Likelihood Reconstruction for Digital Holography with Speckle
2026-02-10 22:31:22Xi Chen, Arian Maleki, Shirin JalaliAbstract
In coherent imaging, speckle is statistically modeled as multiplicative noise, posing a fundamental challenge for image reconstruction. While maximum likelihood estimation (MLE) provides a principled framework for speckle mitigation, its application to coherent imaging system such as digital holography with finite apertures is hindered by the prohibitive cost of high-dimensional matrix inversion, especially at high resolutions. This computational burden has prevented the use of MLE-based reconstruction with physically accurate aperture modeling. In this work, we propose a randomized linear algebra approach that enables scalable MLE optimization without explicit matrix inversions in gradient computation. By exploiting the structural properties of sensing matrix and using conjugate gradient for likelihood gradient evaluation, the proposed algorithm supports accurate aperture modeling without the simplifying assumptions commonly imposed for tractability. We term the resulting method projected gradient descent with Monte Carlo estimation (PGD-MC). The proposed PGD-MC framework (i) demonstrates robustness to diverse and physically accurate aperture models, (ii) achieves substantial improvements in reconstruction quality and computational efficiency, and (iii) scales effectively to high-resolution digital holography. Extensive experiments incorporating three representative denoisers as regularization show that PGD-MC provides a flexible and effective MLE-based reconstruction framework for digital holography with finite apertures, consistently outperforming prior Plug-and-Play model-based iterative reconstruction methods in both accuracy and speed. Our code is available at: this https URL.
Abstract (translated)
在相干成像中，斑点（speckle）被统计建模为乘性噪声，这对图像重建构成了根本性的挑战。最大似然估计（MLE）提供了一种原则化的框架来缓解这种斑点问题，但在具有有限孔径的系统（如数字全息摄影）中的应用却受到高维矩阵求逆成本过高的限制，特别是在高分辨率情况下更是如此。这一计算负担阻碍了物理准确孔径建模下的基于最大似然估计重建方法的应用。 在本文中，我们提出了一种随机线性代数方法，该方法能够在梯度计算过程中不显式使用矩阵求逆的情况下实现可扩展的最大似然优化。通过利用感测矩阵的结构性质，并采用共轭梯度法进行似然梯度评估，所提出的算法支持准确的孔径建模而不需通常为了可处理性而做出的简化假设。我们将该方法称为基于蒙特卡洛估计（Monte Carlo estimation, MC）的投影梯度下降法（projected gradient descent, PGD），简称PGD-MC。 该PGD-MC框架具备以下特点： (i) 对各种物理准确孔径模型具有鲁棒性； (ii) 实现了重建质量和计算效率上的显著改进； (iii) 可以有效地扩展到高分辨率数字全息摄影中。 一系列实验，其中使用了三种代表性的去噪器作为正则化项，展示了PGD-MC为有限孔径数字全息提供了一种灵活且有效的基于最大似然估计的重建框架，并在精度和速度上一直优于以往的Plug-and-Play模型迭代重建方法。我们的代码可在以下链接获取：this https URL.
URL
https://arxiv.org/abs/2602.10344
PDF
-
From Lightweight CNNs to SpikeNets: Benchmarking Accuracy-Energy Tradeoffs with Pruned Spiking SqueezeNet
2026-02-10 12:20:11Radib Bin Kabir, Tawsif Tashwar Dipto, Mehedi Ahamed, Sabbir Ahmed, Md Hasanul KabirAbstract
Spiking Neural Networks (SNNs) are increasingly studied as energy-efficient alternatives to Convolutional Neural Networks (CNNs), particularly for edge intelligence. However, prior work has largely emphasized large-scale models, leaving the design and evaluation of lightweight CNN-to-SNN pipelines underexplored. In this paper, we present the first systematic benchmark of lightweight SNNs obtained by converting compact CNN architectures into spiking networks, where activations are modeled with Leaky-Integrate-and-Fire (LIF) neurons and trained using surrogate gradient descent under a unified setup. We construct spiking variants of ShuffleNet, SqueezeNet, MnasNet, and MixNet, and evaluate them on CIFAR-10, CIFAR-100, and TinyImageNet, measuring accuracy, F1-score, parameter count, computational complexity, and energy consumption. Our results show that SNNs can achieve up to 15.7x higher energy efficiency than their CNN counterparts while retaining competitive accuracy. Among these, the SNN variant of SqueezeNet consistently outperforms other lightweight SNNs. To further optimize this model, we apply a structured pruning strategy that removes entire redundant modules, yielding a pruned architecture, SNN-SqueezeNet-P. This pruned model improves CIFAR-10 accuracy by 6% and reduces parameters by 19% compared to the original SNN-SqueezeNet. Crucially, it narrows the gap with CNN-SqueezeNet, achieving nearly the same accuracy (only 1% lower) but with an 88.1% reduction in energy consumption due to sparse spike-driven computations. Together, these findings establish lightweight SNNs as practical, low-power alternatives for edge deployment, highlighting a viable path toward deploying high-performance, low-power intelligence on the edge.
Abstract (translated)
脉冲神经网络（SNN）作为卷积神经网络（CNN）的节能替代方案，特别是在边缘智能领域中得到了越来越多的关注。然而，先前的研究主要集中在大规模模型上，忽略了轻量级 CNN 到 SNN 转换管道的设计和评估。在这篇论文中，我们介绍了首次系统性地对通过将紧凑型 CNN 架构转换为脉冲网络获得的轻量级 SNN 进行基准测试的结果。在这些转换后的 SNN 中，激活被建模为使用泄露积分放电（LIF）神经元，并采用统一设置下的替代梯度下降法进行训练。 我们构建了 ShuffleNet、SqueezeNet、MnasNet 和 MixNet 的脉冲版本，并针对 CIFAR-10、CIFAR-100 和 TinyImageNet 数据集进行了评估，包括准确率、F1 分数、参数数量、计算复杂性以及能耗等指标。我们的结果显示，在保持竞争力的准确性的前提下，SNN 可以比其 CNN 对应模型高 15.7 倍的能量效率。在这其中，SqueezeNet 的 SNN 变体在所有轻量级 SNN 中表现最优。 为了进一步优化这个模型，我们应用了一种结构化剪枝策略，删除了整个冗余模块，从而得到了一个剪枝架构：SNN-SqueezeNet-P。相比原始的 SNN-SqueezeNet，此剪枝模型将 CIFAR-10 准确率提高了 6%，参数减少了 19%。尤为重要的是，它缩小了与 CNN-SqueezeNet 的差距，在几乎相同的准确度（低仅 1%）下，由于稀疏脉冲驱动的计算，能量消耗降低了 88.1%。 这些发现共同证明轻量级 SNN 在边缘部署中是实用且低能耗的选择，并为在边缘部署高性能、低功耗智能提供了一条可行路径。
URL
https://arxiv.org/abs/2602.09717
PDF
-
Grokking in Linear Models for Logistic Regression
2026-02-09 06:16:43Nataraj Das, Atreya Vedantam, Chandrashekar LakshminarayananAbstract
Grokking, the phenomenon of delayed generalization, is often attributed to the depth and compositional structure of deep neural networks. We study grokking in one of the simplest possible settings: the learning of a linear model with logistic loss for binary classification on data that are linearly (and max margin) separable about the origin. We investigate three testing regimes: (1) test data drawn from the same distribution as the training data, in which case grokking is not observed; (2) test data concentrated around the margin, in which case grokking is observed; and (3) adversarial test data generated via projected gradient descent (PGD) attacks, in which case grokking is also observed. We theoretically show that the implicit bias of gradient descent induces a three-phase learning process-population-dominated, support-vector-dominated unlearning, and support-vector-dominated generalization-during which delayed generalization can arise. Our analysis further relates the emergence of grokking to asymmetries in the data, both in the number of examples per class and in the distribution of support vectors across classes, and yields a characterization of the grokking time. We experimentally validate our theory by planting different distributions of population points and support vectors, and by analyzing accuracy curves and hyperplane dynamics. Overall, our results demonstrate that grokking does not require depth or representation learning, and can emerge even in linear models through the dynamics of the bias term.
Abstract (translated)
“Grokking”现象，即延迟泛化（delayed generalization），通常归因于深度神经网络的深度和组成结构。我们在最简单的可能设置下研究了这种现象：在数据线性可分（且最大间隔可分）的情况下学习具有对数损失函数的线性模型来进行二元分类。我们探讨了三种测试场景：(1) 测试数据与训练数据来自相同的分布，在这种情况下不会观察到Grokking；(2) 测试数据集中在边缘区域，在这种情况下会观察到Grokking；以及(3) 通过投影梯度下降(PGD)攻击生成的对抗性测试数据，在这种情况下也会观察到Grokking。我们理论分析表明，梯度下降所隐含的偏置诱导了一种三阶段的学习过程：整体样本主导、支持向量主导下的去学习和泛化，其中延迟泛化可能在此过程中出现。 我们的研究进一步将Grokking现象与数据中的不对称性关联起来，这些不对称性包括每个类别的示例数量以及各类别之间支点向量（支持向量）分布的差异。我们还给出了关于Grokking时间特性的描述。通过在不同的整体样本和支点向量分布上进行实验，并分析准确率曲线及超平面动态变化，我们的理论得到了验证。 总体而言，研究表明Grokking现象不需要深度网络或表示学习能力，并且即使在线性模型中也能因偏置项的动态而出现。
URL
https://arxiv.org/abs/2602.08302
PDF
