Abstract
We present a novel method for robotic manipulation tasks in human environments that require reasoning about the 3D geometric relationship between a pair of objects. Traditional end-to-end trained policies, which map from pixel observations to low-level robot actions, struggle to reason about complex pose relationships and have difficulty generalizing to unseen object configurations. To address these challenges, we propose a method that learns to reason about the 3D geometric relationship between objects, focusing on the relationship between key parts on one object with respect to key parts on another object. Our standalone model utilizes Weighted SVD to reason about both pose relationships between articulated parts and between free-floating objects. This approach allows the robot to understand the relationship between the oven door and the oven body, as well as the relationship between the lasagna plate and the oven, for example. By considering the 3D geometric relationship between objects, our method enables robots to perform complex manipulation tasks that reason about object-centric representations. We open source the code and demonstrate the results here
Abstract (translated)
我们提出了一种新的方法，用于在人类环境中进行机器人操作任务，该任务需要关于一对物体之间3D几何关系的推理。传统的端到端训练策略，将像素观察映射到低级机器人动作，很难推理关于复杂姿态关系，并且很难推广到未见过的物体配置。为了应对这些挑战，我们提出了一个学习如何推理物体之间3D几何关系的策略，重点关注一个物体上关键部分与另一个物体上关键部分之间的关系。我们的独立模型利用加权SVD来推理关于活动部件之间和自由漂浮物体之间的姿态关系。这种方法允许机器人理解烤箱门和烤箱身体之间的关系，以及披萨盘和烤箱之间的关系，例如。通过考虑物体之间的3D几何关系，我们的方法使机器人能够执行复杂的操作任务，这些任务基于物体中心表示。我们开源了该代码，并在这里展示了结果。
URL
https://arxiv.org/abs/2405.02241
