Abstract
The objective of this work is to evaluate multi-agent artificial intelligence methods when deployed on teams of unmanned surface vehicles (USV) in an adversarial environment. Autonomous agents were evaluated in real-world scenarios using the Aquaticus test-bed, which is a Capture-the-Flag (CTF) style competition involving teams of USV systems. Cooperative teaming algorithms of various foundations in behavior-based optimization and deep reinforcement learning (RL) were deployed on these USV systems in two versus two teams and tested against each other during a competition period in the fall of 2023. Deep reinforcement learning applied to USV agents was achieved via the Pyquaticus test bed, a lightweight gymnasium environment that allows simulated CTF training in a low-level environment. The results of the experiment demonstrate that rule-based cooperation for behavior-based agents outperformed those trained in Deep-reinforcement learning paradigms as implemented in these competitions. Further integration of the Pyquaticus gymnasium environment for RL with MOOS-IvP in terms of configuration and control schema will allow for more competitive CTF games in future studies. As the development of experimental deep RL methods continues, the authors expect that the competitive gap between behavior-based autonomy and deep RL will be reduced. As such, this report outlines the overall competition, methods, and results with an emphasis on future works such as reward shaping and sim-to-real methodologies and extending rule-based cooperation among agents to react to safety and security events in accordance with human experts intent/rules for executing safety and security processes.
Abstract (translated)
本文旨在评估在无人水面车辆（USV）团队中部署多智能体人工智能方法的效果。在2023年秋季的数据竞赛期间，使用Aquaticus测试平台对各种基于行为的优化和深度强化学习（RL）基础的协作算法进行了评估，这些算法被部署在这些USV系统上，以实现双人或双队合作。应用到USV上的深度强化学习通过轻量级的Pyquaticus测试平台实现，这是一个在低级环境中模拟CTF训练的轻量级体育馆环境。实验结果表明，基于行为的智能体代理的合作规则超过了这些竞赛中采用深度强化学习范式的训练结果。进一步研究Pyquaticus体育馆环境和MOOS-IvP之间的配置和控制方案，将在未来的研究中实现更具有竞争力的CTF游戏。随着实验性深度强化学习方法的不断发展，作者预计，基于行为的自主和深度强化学习之间的竞争差距将减少。因此，本报告概述了整个比赛、方法和结果，重点关注未来的研究，例如奖励塑造和模拟-到-现实方法，以及扩展智能体代理之间的规则合作以根据人类专家意图/规则执行安全和安全过程。
URL
https://arxiv.org/abs/2404.17038
