Abstract
We present DexUMI - a data collection and policy learning framework that uses the human hand as the natural interface to transfer dexterous manipulation skills to various robot hands. DexUMI includes hardware and software adaptations to minimize the embodiment gap between the human hand and various robot hands. The hardware adaptation bridges the kinematics gap using a wearable hand exoskeleton. It allows direct haptic feedback in manipulation data collection and adapts human motion to feasible robot hand motion. The software adaptation bridges the visual gap by replacing the human hand in video data with high-fidelity robot hand inpainting. We demonstrate DexUMI's capabilities through comprehensive real-world experiments on two different dexterous robot hand hardware platforms, achieving an average task success rate of 86%.
Abstract (translated)
我们介绍了DexUMI——一个数据采集和策略学习框架，该框架利用人手作为自然接口，将灵巧的操作技能转移到各种机器人手中。DexUMI 包括硬件和软件的适应性调整，以最小化人手与不同机器人手之间的实体差距。硬件方面的调整通过穿戴式手外骨骼来弥补运动学上的差异，它允许在操作数据收集过程中直接进行触觉反馈，并将人类的动作转换为机器人手可执行的动作。软件方面的调整则通过在视频数据中用高保真的机器人手图像替换人手来弥合视觉差距。我们通过在两种不同的灵巧机器人手上进行了全面的现实世界实验，展示了DexUMI 的功能，实现了86% 的平均任务成功率。
URL
https://arxiv.org/abs/2505.21864
