Abstract
The ability to leverage shared behaviors between tasks is critical for sample-efficient multi-task reinforcement learning (MTRL). While prior methods have primarily explored parameter and data sharing, direct behavior-sharing has been limited to task families requiring similar behaviors. Our goal is to extend the efficacy of behavior-sharing to more general task families that could require a mix of shareable and conflicting behaviors. Our key insight is an agent's behavior across tasks can be used for mutually beneficial exploration. To this end, we propose a simple MTRL framework for identifying shareable behaviors over tasks and incorporating them to guide exploration. We empirically demonstrate how behavior sharing improves sample efficiency and final performance on manipulation and navigation MTRL tasks and is even complementary to parameter sharing. Result videos are available at this https URL.
Abstract (translated)
利用任务之间的共享行为对于高效样本学习的多任务强化学习(MTRL)至关重要。先前的方法主要探索参数和数据共享,直接的行为共享仅局限于需要类似行为的任务家族。我们的目标是将行为共享扩展到更广泛的任务家族,可能需要可共享和冲突行为的混合。我们的关键发现是,任务间的行为可以用于相互有益的探索。为此,我们提出了一个简单的MTRL框架,用于确定任务间可共享的行为,并将它们用于指导探索。我们的经验证明,行为共享如何改善操作和导航MTRL任务的效率并最终提高其表现,甚至与参数共享互为补充。结果视频可在此https URL中找到。
URL
https://arxiv.org/abs/2302.00671
