Abstract
Recent advancements in Large Language Models (LLMs) have revealed a significant performance gap between closed-source and open-source models, particularly in tasks requiring complex reasoning and precise instruction following. This paper introduces ReasonBridge, a methodology that efficiently transfers reasoning capabilities from powerful closed-source to open-source models through a novel hierarchical knowledge distillation framework. We develop a tailored dataset Reason1K with only 1,000 carefully curated reasoning traces emphasizing difficulty, diversity, and quality. These traces are filtered from across multiple domains using a structured multi-criteria selection algorithm. Our transfer learning approach incorporates: (1) a hierarchical distillation process capturing both strategic abstraction and tactical implementation patterns, (2) a sparse reasoning-focused adapter architecture requiring only 0.3% additional trainable parameters, and (3) a test-time compute scaling mechanism using guided inference interventions. Comprehensive evaluations demonstrate that ReasonBridge improves reasoning capabilities in open-source models by up to 23% on benchmark tasks, significantly narrowing the gap with closed-source models. Notably, the enhanced Qwen2.5-14B outperforms Claude-Sonnet3.5 on MATH500 and matches its performance on competition-level AIME problems. Our methodology generalizes effectively across diverse reasoning domains and model architectures, establishing a sample-efficient approach to reasoning enhancement for instruction following.
Abstract (translated)
近期在大型语言模型（LLMs）领域的进展揭示了闭源模型和开源模型之间存在显著的性能差距，尤其是在需要复杂推理和精准指令执行的任务中。本文介绍了ReasonBridge，一种通过新颖的分层知识蒸馏框架高效地将强大闭源模型的推理能力转移到开源模型的方法论。我们开发了一个特制的数据集Reason1K，仅包含精心策划的1,000个推理痕迹，这些痕迹强调难度、多样性和质量，并且是从多个领域中筛选出来的，使用的是结构化多准则选择算法。我们的迁移学习方法包括： 1. 一种分层蒸馏过程，捕捉策略抽象和战术实施模式。 2. 一个专注于稀疏推理的适配器架构，只需要增加0.3%的可训练参数。 3. 在测试时采用计算缩放机制，使用引导式推断干预。 全面评估表明，ReasonBridge在基准任务上将开源模型的推理能力提高了高达23%，显著缩小了与闭源模型之间的差距。特别值得注意的是，经过增强的Qwen2.5-14B在MATH500和竞赛级别的AIME问题上表现优于Claude-Sonnet3.5，并且在后者的问题上与其性能相当。 我们的方法论能够在广泛的推理领域和不同架构的模型中有效推广，建立了一种高效的推理增强方法来改进指令遵循。
URL
https://arxiv.org/abs/2506.22865
