Abstract
Recent advancements in large reasoning models (LRMs) have significantly enhanced language models' capabilities in complex problem-solving by emulating human-like deliberative thinking. However, these models often exhibit overthinking (i.e., the generation of unnecessarily verbose and redundant content), which hinders efficiency and inflates inference cost. In this work, we explore the representational and behavioral origins of this inefficiency, revealing that LRMs inherently possess the capacity for more concise reasoning. Empirical analyses show that correct reasoning paths vary significantly in length, and the shortest correct responses often suffice, indicating untapped efficiency potential. Exploiting these findings, we propose two lightweight methods to enhance LRM efficiency. First, we introduce Efficiency Steering, a training-free activation steering technique that modulates reasoning behavior via a single direction in the model's representation space. Second, we develop Self-Rewarded Efficiency RL, a reinforcement learning framework that dynamically balances task accuracy and brevity by rewarding concise correct solutions. Extensive experiments on seven LRM backbones across multiple mathematical reasoning benchmarks demonstrate that our methods significantly reduce reasoning length while preserving or improving task performance. Our results highlight that reasoning efficiency can be improved by leveraging and guiding the intrinsic capabilities of existing models in a self-guided manner.
Abstract (translated)
最近在大规模推理模型（LRMs）方面的进展显著增强了语言模型解决复杂问题的能力，通过模拟人类的审慎思考。然而，这些模型经常表现出过度思考的现象，即生成冗长且不必要的内容，这会降低效率并增加推断成本。在这项工作中，我们探索了这种低效性的表征和行为根源，并揭示出LRMs本身就具备进行更加简洁推理的能力。实证分析表明，正确的推理路径长度差异很大，而最短的正确答案通常就足够了，这意味着存在未被开发的效率潜力。 基于这些发现，我们提出了两种轻量级方法来提高LRM的效率。首先，我们引入了“Efficiency Steering”，这是一种无须重新训练的激活控制技术，通过在模型表征空间中的单一方向调节推理行为。其次，我们开发了一种名为“Self-Rewarded Efficiency RL”的强化学习框架，该框架动态平衡任务准确性和简洁性，通过奖励简短且正确的解决方案来实现。 我们在七个LRM基础架构上进行了广泛的实验，并针对多个数学推理基准测试展示了我们的方法显著减少了推理长度，同时保持甚至提高了任务性能。我们的结果表明，可以通过引导现有模型的内在能力来自我指导地改进推理效率。
URL
https://arxiv.org/abs/2506.15647
