Abstract
Reinforcement Learning (RL) provides a framework in which agents can be trained, via trial and error, to solve complex decision-making problems. Learning with little supervision causes RL methods to require large amounts of data, which renders them too expensive for many applications (e.g. robotics). By reusing knowledge from a different task, knowledge transfer methods present an alternative to reduce the training time in RL. Given how severe data scarcity can be, there has been a growing interest for methods capable of transferring knowledge across different domains (i.e. problems with different representation) due to the flexibility they offer. This review presents a unifying analysis of methods focused on transferring knowledge across different domains. Through a taxonomy based on a transfer-approach categorization, and a characterization of works based on their data-assumption requirements, the objectives of this article are to 1) provide a comprehensive and systematic revision of knowledge transfer methods for the cross-domain RL setting, 2) categorize and characterize these methods to provide an analysis based on relevant features such as their transfer approach and data requirements, and 3) discuss the main challenges regarding cross-domain knowledge transfer, as well as ideas of future directions worth exploring to address these problems.
Abstract (translated)
强化学习（RL）提供了一个框架，让智能体通过尝试和错误，训练来解决复杂的决策问题。少量的监督学习导致RL方法需要大量数据，这使得它们对许多应用（如机器人学）来说过于昂贵。通过将不同任务中的知识进行重用，知识迁移方法提供了一种减少RL培训时间的方法。由于数据稀缺的严重程度，人们对能够在不同领域之间转移知识的方法产生了浓厚兴趣，因为它们提供了灵活性。本文对关注跨领域知识传递的方法进行了统一分析。通过基于迁移方法分类的树状结构，以及根据数据假设要求对作品进行特征描述，本文的目的是1)提供一个全面的关于跨领域RL设置中的知识传递方法的全面和系统的回顾，2)对这类方法进行分类和定性，以便根据其迁移方法和数据需求提供有关其相关特征的分析，3)讨论跨领域知识传递的主要挑战以及值得探索的未来方向来解决这些问题。
URL
https://arxiv.org/abs/2404.17687
