Abstract
In recent years, multi-agent reinforcement learning algorithms have made significant advancements in diverse gaming environments, leading to increased interest in the broader application of such techniques. To address the prevalent challenge of partial observability, communication-based algorithms have improved cooperative performance through the sharing of numerical embedding between agents. However, the understanding of the formation of collaborative mechanisms is still very limited, making designing a human-understandable communication mechanism a valuable problem to address. In this paper, we propose a novel multi-agent reinforcement learning algorithm that embeds large language models into agents, endowing them with the ability to generate human-understandable verbal communication. The entire framework has a message module and an action module. The message module is responsible for generating and sending verbal messages to other agents, effectively enhancing information sharing among agents. To further enhance the message module, we employ a teacher model to generate message labels from the global view and update the student model through Supervised Fine-Tuning (SFT). The action module receives messages from other agents and selects actions based on current local observations and received messages. Experiments conducted on the Overcooked game demonstrate our method significantly enhances the learning efficiency and performance of existing methods, while also providing an interpretable tool for humans to understand the process of multi-agent cooperation.
Abstract (translated)
近年来，多智能体强化学习算法在各种游戏环境中取得了显著的进步，导致人们对这些技术的更广泛应用产生了浓厚兴趣。为了应对普遍存在的部分可观察性挑战，基于通信的算法通过共享代理之间的数值嵌入来提高合作性能。然而，对协作机制的理解仍然非常有限，使得设计一个可被人类理解的信息传输机制成为一个有价值的问题。在本文中，我们提出了一个新颖的多智能体强化学习算法，将大型语言模型嵌入到代理中，使它们具有生成人类可理解口头交流的能力。整个框架包括消息模块和动作模块。消息模块负责生成和发送口头信息给其他代理，有效提高了代理之间的信息共享。为了进一步提高消息模块，我们使用教师模型从全局视角生成消息标签，并通过监督微调（SFT）更新学生模型。在Overcooked游戏中进行实验证明，我们的方法显著增强了现有方法的学习效率和性能，并为人类理解多智能体合作的过程提供了一个可解释的工具。
URL
https://arxiv.org/abs/2404.17780
