Abstract
Large Language Models (LLMs) are increasingly being deployed as autonomous agents in real-world environments. As these deployments scale, multi-agent interactions become inevitable, making it essential to understand strategic behavior in such systems. A central open question is whether LLM agents, like reinforcement learning agents, can shape the learning dynamics and influence the behavior of others through interaction alone. In this paper, we present the first investigation of opponent shaping (OS) with LLM-based agents. Existing OS algorithms cannot be directly applied to LLMs, as they require higher-order derivatives, face scalability constraints, or depend on architectural components that are absent in transformers. To address this gap, we introduce ShapeLLM, an adaptation of model-free OS methods tailored for transformer-based agents. Using ShapeLLM, we examine whether LLM agents can influence co-players' learning dynamics across diverse game-theoretic environments. We demonstrate that LLM agents can successfully guide opponents toward exploitable equilibria in competitive games (Iterated Prisoner's Dilemma, Matching Pennies, and Chicken) and promote coordination and improve collective welfare in cooperative games (Iterated Stag Hunt and a cooperative version of the Prisoner's Dilemma). Our findings show that LLM agents can both shape and be shaped through interaction, establishing opponent shaping as a key dimension of multi-agent LLM research.
Abstract (translated)
大型语言模型（LLMs）正越来越多地被部署为现实世界环境中的自主代理。随着这些部署规模的扩大，多代理交互变得不可避免，因此理解此类系统中的策略行为至关重要。一个核心的开放性问题是，像强化学习代理一样，LLM代理是否可以通过单独互动来塑造学习动态并影响他人的行为。在本文中，我们首次对基于LLM的对手塑形（OS）进行了研究。现有的OS算法不能直接应用于LLMs，因为它们需要更高阶的导数、面临可扩展性限制或依赖于转换器架构所不具备的组件。为了解决这一差距，我们引入了ShapeLLM，这是一种针对基于变压器代理的无模型OS方法的改进版。使用ShapeLLM，我们研究了LLM代理是否可以在各种博弈论环境中影响对手的学习动态。我们的研究表明，LLM代理能够成功引导竞争对手走向可被利用的均衡点（在竞争性游戏中如反复囚徒困境、Matching Pennies和Chicken），并在合作游戏中促进协调并提升集体福利（如重复猎鹿游戏及一个合作版本的囚徒困境）。研究结果表明，LLM代理可以相互塑造并通过互动来进行影响，这将对手塑形确立为多代理LLM研究的关键维度。
URL
https://arxiv.org/abs/2510.08255
