Abstract
Recent advancements in large language models (LLMs) have led to significant successes across various applications, where the most noticeable is to a series of emerging capabilities, particularly in the areas of In-Context Learning (ICL) and Chain-of-Thought (CoT). To better understand and control model performance, many studies have begun investigating the underlying causes of these phenomena and their impact on task outcomes. However, existing explanatory frameworks predominantly focus on isolating and explaining ICL and CoT independently, leading to an incomplete understanding of their combined influence on model performance. To address this gap, we propose the Electronic Circuit Model (ECM), which provides a foundation for developing scalable, learnable policies and improving the management of AI-generated content. Specifically, ECM conceptualizes model behavior as an electronic circuit: ICL is represented as semantic magnetic field to providing an additional voltage following Faraday's Law, while CoT is modeled as series resistors to constrain the model output performance following Ohm's Law. Experimental results demonstrate that the ECM effectively predicts and explains LLM performance across a variety of prompting strategies. Furthermore, we apply ECM to advanced reasoning strategy optimization on a series of tasks, such as the International Olympiad in Informatics (IOI) and the International Mathematical Olympiad (IMO), achieving competitive performance that surpasses nearly 80% of top human competitors.
Abstract (translated)
最近在大型语言模型（LLMs）方面取得的进展已经在各种应用中取得了显著的成功，其中最引人注目的是新兴能力的发展，特别是在上下文学习（ICL）和链式思维（CoT）领域。为了更好地理解和控制模型性能，许多研究开始调查这些现象的根本原因及其对任务结果的影响。然而，现有的解释框架主要集中在单独隔离并解释ICL和CoT上，这导致了对其结合影响模型表现的理解不完整。为了解决这一缺口，我们提出了电子电路模型（ECM），它提供了一个开发可扩展、可学习的政策以及改进AI生成内容管理的基础。具体来说，ECM将模型行为比喻成一个电子电路：ICL被表示为通过法拉第定律提供的语义磁场来增加额外电压，而CoT则被建模为串联电阻以限制模型输出性能遵循欧姆定律。 实验结果表明，ECM能够有效预测和解释各种提示策略下的LLM表现。此外，我们还将ECM应用于一系列任务（如国际信息学奥林匹克竞赛(IOI) 和 国际数学奥林匹克(IMO)）的高级推理策略优化上，在这些任务中实现了超越近80%顶级人类竞争对手的竞争力性能。
URL
https://arxiv.org/abs/2502.03325
