Abstract
Replicating the remarkable athleticism seen in animals has long been a challenge in robotics control. Although Reinforcement Learning (RL) has demonstrated significant progress in dynamic legged locomotion control, the substantial sim-to-real gap often hinders the real-world demonstration of truly dynamic movements. We propose a new framework to mitigate this gap through frequency-domain analysis-based impedance matching between simulated and real robots. Our framework offers a structured guideline for parameter selection and the range for dynamics randomization in simulation, thus facilitating a safe sim-to-real transfer. The learned policy using our framework enabled jumps across distances of 55 cm and heights of 38 cm. The results are, to the best of our knowledge, one of the highest and longest running jumps demonstrated by an RL-based control policy in a real quadruped robot. Note that the achieved jumping height is approximately 85% of that obtained from a state-of-the-art trajectory optimization method, which can be seen as the physical limit for the given robot hardware. In addition, our control policy accomplished stable walking at speeds up to 2 m/s in the forward and backward directions, and 1 m/s in the sideway direction.
Abstract (translated)
复制动物在运动中的惊人 athletic 性一直是一个挑战，尤其是在机器人控制领域。虽然强化学习 (RL) 在动态腿履带运动控制方面取得了显著的进步，但巨大的模拟与现实之间的差距通常会阻碍在现实世界中真正动态运动的演示。我们提出了一种新的框架，通过基于频域分析的模拟与现实机器人之间的阻尼匹配来缓解这个差距。我们的框架为参数选择和动态随机化在模拟中的范围提供了结构化的指导，从而促进了安全的模拟到实体的转移。使用我们框架学习到的策略，跳跃距离达到了55厘米，高度达到了38厘米。据我们所知，这是基于 RL 的控制策略在实心四足机器人中实现的最高和最长的跳跃。需要注意的是，所达到的跳跃高度大约是先进轨迹优化方法得到的结果的85%，可以看出这是给定机器人硬件的物理极限。此外，我们的控制策略在前进和后退方向上实现了稳定的步行，速度达到2米/秒，而在侧面方向上实现了1米/秒的步行。
URL
https://arxiv.org/abs/2404.15096
