Abstract
In various real-world scenarios, interactions among agents often resemble the dynamics of general-sum games, where each agent strives to optimize its own utility. Despite the ubiquitous relevance of such settings, decentralized machine learning algorithms have struggled to find equilibria that maximize individual utility while preserving social welfare. In this paper we introduce Learning with Opponent Q-Learning Awareness (LOQA), a novel, decentralized reinforcement learning algorithm tailored to optimizing an agent's individual utility while fostering cooperation among adversaries in partially competitive environments. LOQA assumes the opponent samples actions proportionally to their action-value function Q. Experimental results demonstrate the effectiveness of LOQA at achieving state-of-the-art performance in benchmark scenarios such as the Iterated Prisoner's Dilemma and the Coin Game. LOQA achieves these outcomes with a significantly reduced computational footprint, making it a promising approach for practical multi-agent applications.
Abstract (translated)
在各种现实场景中,智能体之间的交互通常类似于一般博弈论中的动态,其中每个智能体都试图最大化自己的效用。尽管这种设置的普遍性至关重要,但分散式机器学习算法在保持社会福利的同时找到最优个人效用平衡方面遇到了困难。在本文中,我们引入了学习与对手Q学习意识(LOQA)算法,这是一种专为在部分竞争环境中促进对抗体合作而设计的分布式强化学习算法。LOQA假定对抗体样本的动作比例与其价值函数Q成比例。实验结果表明,LOQA在标准场景(如迭代囚徒困境和硬币游戏)中的表现已经达到最先进的水平。LOQA通过显著减少计算负担实现了这些成果,使得它成为实用多智能体应用程序的有前景的方法。
URL
https://arxiv.org/abs/2405.01035
