Abstract
We investigate policy transfer using image-to-semantics translation to mitigate learning difficulties in vision-based robotics control agents. This problem assumes two environments: a simulator environment with semantics, that is, low-dimensional and essential information, as the state space, and a real-world environment with images as the state space. By learning mapping from images to semantics, we can transfer a policy, pre-trained in the simulator, to the real world, thereby eliminating real-world on-policy agent interactions to learn, which are costly and risky. In addition, using image-to-semantics mapping is advantageous in terms of the computational efficiency to train the policy and the interpretability of the obtained policy over other types of sim-to-real transfer strategies. To tackle the main difficulty in learning image-to-semantics mapping, namely the human annotation cost for producing a training dataset, we propose two techniques: pair augmentation with the transition function in the simulator environment and active learning. We observed a reduction in the annotation cost without a decline in the performance of the transfer, and the proposed approach outperformed the existing approach without annotation.
Abstract (translated)
我们研究使用图像到语义翻译来解决基于视觉机器人控制 agents 的学习困难。这个问题假设了两个环境:一个语义环境,即低维度和关键信息,作为状态空间,另一个现实世界环境,以图像作为状态空间。通过学习从图像到语义的映射,我们可以将已经在模拟器中训练的政策转移到现实世界,从而消除现实世界的政策执行与学习之间的实际执行交互,这些交互昂贵且风险高。此外,使用图像到语义映射在训练政策效率和获得政策的解释性方面优于其他类型的模拟器到现实的传输策略。为了应对学习图像到语义映射的主要困难,即产生训练数据的人类标注成本,我们提出了两种方法:在模拟器环境中的配对增强和主动学习。我们观察到标注成本下降了,但传输表现并没有下降,而 proposed 方法超越了没有标注的方法。
URL
https://arxiv.org/abs/2301.13343
