Abstract
The escalating scale of Large Language Models (LLMs) necessitates efficient adaptation techniques. Model merging has gained prominence for its efficiency and controllability. However, existing merging techniques typically serve as post-hoc refinements or focus on mitigating task interference, often failing to capture the dynamic optimization benefits of supervised fine-tuning (SFT). In this work, we propose Streaming Merging, an innovative model updating paradigm that conceptualizes merging as an iterative optimization process. Central to this paradigm is \textbf{ARM} (\textbf{A}ctivation-guided \textbf{R}otation-aware \textbf{M}erging), a strategy designed to approximate gradient descent dynamics. By treating merging coefficients as learning rates and deriving rotation vectors from activation subspaces, ARM effectively steers parameter updates along data-driven trajectories. Unlike conventional linear interpolation, ARM aligns semantic subspaces to preserve the geometric structure of high-dimensional parameter evolution. Remarkably, ARM requires only early SFT checkpoints and, through iterative merging, surpasses the fully converged SFT model. Experimental results across model scales (1.7B to 14B) and diverse domains (e.g., math, code) demonstrate that ARM can transcend converged checkpoints. Extensive experiments show that ARM provides a scalable and lightweight framework for efficient model adaptation.
Abstract (translated)
大型语言模型（LLM）规模的不断扩大要求更高效的适应技术。模型合并因其高效性和可控性而备受关注。然而，现有的合并技术通常作为事后优化或专注于减少任务干扰，并且往往未能捕捉到有监督微调（SFT）中的动态优化收益。在这项工作中，我们提出了流式合并（Streaming Merging），这是一种创新的模型更新范式，它将合并视为一个迭代优化过程。这一范式的中心是**ARM**（激活指导的旋转感知合并），这是一种旨在近似梯度下降动力学的战略。通过将合并系数视为学习率并从激活子空间中推导出旋转向量，ARM能够引导参数沿数据驱动轨迹进行更新。不同于传统的线性插值方法，ARM对齐语义子空间以保持高维参数演化的几何结构。值得注意的是，ARM只需要早期的SFT检查点，并通过迭代合并超越了完全收敛的SFT模型。跨不同规模（从17亿到140亿）和多样领域（如数学、代码等）进行实验的结果表明，ARM能够超越已达成共识的检查点。广泛的实验显示，ARM提供了一个可扩展且轻量级的框架，用于高效地调整模型适应性。
URL
https://arxiv.org/abs/2602.03237
