Abstract
The significance of network structures in promoting group cooperation within social dilemmas has been widely recognized. Prior studies attribute this facilitation to the assortment of strategies driven by spatial interactions. Although reinforcement learning has been employed to investigate the impact of dynamic interaction on the evolution of cooperation, there remains a lack of understanding about how agents develop neighbour selection behaviours and the formation of strategic assortment within an explicit interaction structure. To address this, our study introduces a computational framework based on multi-agent reinforcement learning in the spatial Prisoner's Dilemma game. This framework allows agents to select dilemma strategies and interacting neighbours based on their long-term experiences, differing from existing research that relies on preset social norms or external incentives. By modelling each agent using two distinct Q-networks, we disentangle the coevolutionary dynamics between cooperation and interaction. The results indicate that long-term experience enables agents to develop the ability to identify non-cooperative neighbours and exhibit a preference for interaction with cooperative ones. This emergent self-organizing behaviour leads to the clustering of agents with similar strategies, thereby increasing network reciprocity and enhancing group cooperation.
Abstract (translated)
社交困境中网络结构在促进群体合作方面的 significance已经被广泛认可。之前的研究将这种促进归因于空间交互驱动的策略多样化。尽管使用强化学习研究了动态交互对合作演化影响的效应，但关于如何在显式互动结构中发展邻居选择行为和战略组合仍然存在理解不足。为了应对这个问题，我们的研究基于多代理强化学习在囚徒困境游戏中的框架。这个框架允许代理根据其长期经验选择困境策略和互动邻居，与现有的研究不同，后者依赖于预设的社会规范或外部激励。通过使用两个不同的Q网络来建模每个代理，我们解开了合作和互动之间的协同进化动态。结果显示，长期经验使代理能够发展出识别非合作邻居和偏好与积极合作者互动的能力。这种自组织行为导致具有相似策略的代理聚类，从而增加网络互惠和提高群体合作。
URL
https://arxiv.org/abs/2405.02654
