Abstract
We study the problem of object retrieval in scenarios where visual sensing is absent, object shapes are unknown beforehand and objects can move freely, like grabbing objects out of a drawer. Successful solutions require localizing free objects, identifying specific object instances, and then grasping the identified objects, only using touch feedback. Unlike vision, where cameras can observe the entire scene, touch sensors are local and only observe parts of the scene that are in contact with the manipulator. Moreover, information gathering via touch sensors necessitates applying forces on the touched surface which may disturb the scene itself. Reasoning with touch, therefore, requires careful exploration and integration of information over time -- a challenge we tackle. We present a system capable of using sparse tactile feedback from fingertip touch sensors on a dexterous hand to localize, identify and grasp novel objects without any visual feedback. Videos are available at this https URL.
Abstract (translated)
我们研究的是视觉感知不存在、物体形状未知且能够自由移动的场景，例如取出抽屉中的物品。成功的解决方案需要定位自由物品、识别特定的实例并用手抓住被识别的物品，仅使用触摸反馈。与视觉不同的是，相机只能观察整个场景，触摸传感器是局部的，只能观察与操纵器接触的部分场景。此外，通过触摸传感器收集信息需要在触摸表面上施加力量，这可能会干扰场景本身。因此，与触摸进行推理需要仔细探索和整合信息——这是一个我们挑战的问题。我们提出了一种系统，它能够利用手指尖端触摸传感器提供的稀疏触觉反馈，在没有任何视觉反馈的情况下定位、识别并抓住新物品。视频资源可在以下 https URL 中找到。
URL
https://arxiv.org/abs/2303.13482
