Abstract
In this work, we introduce the EyeSight Hand, a novel 7 degrees of freedom (DoF) humanoid hand featuring integrated vision-based tactile sensors tailored for enhanced whole-hand manipulation. Additionally, we introduce an actuation scheme centered around quasi-direct drive actuation to achieve human-like strength and speed while ensuring robustness for large-scale data collection. We evaluate the EyeSight Hand on three challenging tasks: bottle opening, plasticine cutting, and plate pick and place, which require a blend of complex manipulation, tool use, and precise force application. Imitation learning models trained on these tasks, with a novel vision dropout strategy, showcase the benefits of tactile feedback in enhancing task success rates. Our results reveal that the integration of tactile sensing dramatically improves task performance, underscoring the critical role of tactile information in dexterous manipulation.
Abstract (translated)
在这项工作中，我们引入了EyeSight Hand，这是一种新型的具有7个自由度（DoF）的人形手，其集成式视觉传感器专为增强整体手操作而设计。此外，我们还引入了一种以准直驱动作为核心的施力方案，以实现类似于人类的力量和速度，同时确保在大规模数据收集过程中的稳健性。我们对EyeSight Hand在三个具有挑战性的任务上的表现进行了评估：开瓶子、塑料刀切割和托盘装配，这些任务需要复杂的操作、工具使用和精确的力量施加。在这些任务上，通过训练模仿学习模型并采用新的视觉 dropout策略，展示了触觉反馈在提高任务成功率方面的优势。我们的结果表明，集成式触觉传感显著提高了任务性能，突显了触觉信息在灵巧操作中的关键作用。
URL
https://arxiv.org/abs/2408.06265
