Abstract
Open-sourced, user-friendly tools form the bedrock of scientific advancement across disciplines. The widespread adoption of data-driven learning has led to remarkable progress in multi-fingered dexterity, bimanual manipulation, and applications ranging from logistics to home robotics. However, existing data collection platforms are often proprietary, costly, or tailored to specific robotic morphologies. We present OPEN TEACH, a new teleoperation system leveraging VR headsets to immerse users in mixed reality for intuitive robot control. Built on the affordable Meta Quest 3, which costs $500, OPEN TEACH enables real-time control of various robots, including multi-fingered hands and bimanual arms, through an easy-to-use app. Using natural hand gestures and movements, users can manipulate robots at up to 90Hz with smooth visual feedback and interface widgets offering closeup environment views. We demonstrate the versatility of OPEN TEACH across 38 tasks on different robots. A comprehensive user study indicates significant improvement in teleoperation capability over the AnyTeleop framework. Further experiments exhibit that the collected data is compatible with policy learning on 10 dexterous and contact-rich manipulation tasks. Currently supporting Franka, xArm, Jaco, and Allegro platforms, OPEN TEACH is fully open-sourced to promote broader adoption. Videos are available at this https URL.
Abstract (translated)
开源、用户友好的工具是跨学科科学进步的基础。数据驱动的学习的广泛采用导致多指灵巧、双臂操作和应用范围从物流到家庭机器人学的显著进步。然而，现有的数据收集平台通常都是专有、昂贵或针对特定机器人形态定制的。我们介绍了一种名为OPEN TEACH的新遥控系统，利用VR头盔实现用户在混合现实中的直观机器人控制。该系统基于价格实惠的Meta Quest 3，售价500美元。通过易用的应用程序，OPEN TEACH可以实时控制各种机器人，包括多指手和双臂，从而实现高效的操作。用户可以使用自然手势和动作操纵机器人，并实现高达90Hz的流畅视觉反馈和界面插件提供近距离环境观察。我们在不同机器人上展示了OPEN TEACH的多样性。全面的用户研究显示，与AnyTeleop框架相比，遥控能力得到了显著提高。进一步的实验表明，收集的数据与在10个多指和触觉丰富的操作任务上进行策略学习是兼容的。目前，OPEN TEACH支持Franka、xArm、Jaco和Allegro平台，完全开源以促进更广泛的采用。视频可在此链接观看。
URL
https://arxiv.org/abs/2403.07870
