Abstract
This work introduces DiffuseLoco, a framework for training multi-skill diffusion-based policies for dynamic legged locomotion from offline datasets, enabling real-time control of diverse skills on robots in the real world. Offline learning at scale has led to breakthroughs in computer vision, natural language processing, and robotic manipulation domains. However, scaling up learning for legged robot locomotion, especially with multiple skills in a single policy, presents significant challenges for prior online reinforcement learning methods. To address this challenge, we propose a novel, scalable framework that leverages diffusion models to directly learn from offline multimodal datasets with a diverse set of locomotion skills. With design choices tailored for real-time control in dynamical systems, including receding horizon control and delayed inputs, DiffuseLoco is capable of reproducing multimodality in performing various locomotion skills, zero-shot transfer to real quadrupedal robots, and it can be deployed on edge computing devices. Furthermore, DiffuseLoco demonstrates free transitions between skills and robustness against environmental variations. Through extensive benchmarking in real-world experiments, DiffuseLoco exhibits better stability and velocity tracking performance compared to prior reinforcement learning and non-diffusion-based behavior cloning baselines. The design choices are validated via comprehensive ablation studies. This work opens new possibilities for scaling up learning-based legged locomotion controllers through the scaling of large, expressive models and diverse offline datasets.
Abstract (translated)
本文介绍了一种名为DiffuseLoco的多技能扩散模型的训练框架，用于从离线数据中训练多技能动态腿部运动策略，实现对现实世界中机器人的实时控制。大规模的离线学习在计算机视觉、自然语言处理和机器人操作领域取得了突破。然而，对于具有单一策略的机器人运动控制，尤其是在多个技能的情况下，扩展学习带来了巨大的挑战，对于先前的在线强化学习方法而言。为解决这个问题，我们提出了一个新型的、可扩展的框架，它利用扩散模型从离线多模态数据中直接学习，具有多样化的运动技能。通过针对动态系统进行设计的决策，包括后退视野控制和延迟输入，DiffuseLoco能够复制各种运动技能，实现零散地将机器人转移到真实四足机器人，并且可以部署在边缘计算设备上。此外，DiffuseLoco展示了技能之间的自由转换和对抗环境变化的能力。通过在现实世界实验中进行广泛的基准测试，DiffuseLoco与先前的强化学习和基于非扩散模型的行为克隆基线相比，表现出更好的稳定性和速度跟踪性能。通过全面的消融分析验证了设计选择。这项工作为通过扩展基于学习的机器人运动控制器打开了新的可能性，通过扩展大型、表现力强的模型和多样化的离线数据。
URL
https://arxiv.org/abs/2404.19264
