Abstract
Domain randomization is a popular technique for improving domain transfer, often used in a zero-shot setting when the target domain is unknown or cannot easily be used for training. In this work, we empirically examine the effects of domain randomization on agent generalization. Our experiments show that domain randomization may lead to suboptimal, high-variance policies, which we attribute to the uniform sampling of environment parameters. We propose Active Domain Randomization, a novel algorithm that learns a parameter sampling strategy. Our method looks for the most informative environment variations within the given randomization ranges by leveraging the discrepancies of policy rollouts in randomized and reference environment instances. We find that training more frequently on these instances leads to better overall agent generalization. In addition, when domain randomization and policy transfer fail, Active Domain Randomization offers more insight into the deficiencies of both the chosen parameter ranges and the learned policy, allowing for more focused debugging. Our experiments across various physics-based simulated and a real-robot task show that this enhancement leads to more robust, consistent policies.
Abstract (translated)
域随机化是一种常用的改进域传输的技术，通常用于目标域未知或不易用于训练的零触发设置中。在这项工作中，我们实证研究了领域随机化对代理泛化的影响。我们的实验表明，区域随机化可能导致次优、高方差策略，这归因于环境参数的均匀抽样。提出了一种新的学习参数抽样策略的主动域随机化算法。我们的方法通过利用随机和参考环境实例中策略推广的差异来寻找给定随机范围内信息最丰富的环境变化。我们发现，对这些实例进行更频繁的培训会导致更好的总体代理泛化。此外，当域随机化和策略传输失败时，主动域随机化可以更深入地了解所选参数范围和所学策略的缺陷，从而使调试更加集中。我们对各种基于物理的模拟和实际机器人任务进行的实验表明，这种增强会导致更健壮、一致的策略。
URL
https://arxiv.org/abs/1904.04762
