Abstract
The ability for robots to perform efficient and zero-shot grasping of object parts is crucial for practical applications and is becoming prevalent with recent advances in Vision-Language Models (VLMs). To bridge the 2D-to-3D gap for representations to support such a capability, existing methods rely on neural fields (NeRFs) via differentiable rendering or point-based projection methods. However, we demonstrate that NeRFs are inappropriate for scene changes due to their implicitness and point-based methods are inaccurate for part localization without rendering-based optimization. To amend these issues, we propose GraspSplats. Using depth supervision and a novel reference feature computation method, GraspSplats generates high-quality scene representations in under 60 seconds. We further validate the advantages of Gaussian-based representation by showing that the explicit and optimized geometry in GraspSplats is sufficient to natively support (1) real-time grasp sampling and (2) dynamic and articulated object manipulation with point trackers. With extensive experiments on a Franka robot, we demonstrate that GraspSplats significantly outperforms existing methods under diverse task settings. In particular, GraspSplats outperforms NeRF-based methods like F3RM and LERF-TOGO, and 2D detection methods.
Abstract (translated)
机器人能够高效且无 shot 地抓取物体部件的能力对实际应用至关重要，并且随着 Vision-Language Models (VLMs) 的最新进展而变得普遍。为了在表示中实现这种能力，现有方法依赖于神经场 (NeRFs) 通过不同的渲染或基于点的投影方法。然而，我们证明了 NeRFs 不适用于场景变化，因为它们的隐含性和基于点的方法在没有渲染优化时无法准确抓取部件。为了弥补这些问题，我们提出了 GraspSplats。通过深度监督和一种新颖的参考特征计算方法，GraspSplats 在不到 60 秒的时间内生成了高质量的场景表示。我们进一步验证了基于高斯表示的优越性，通过表明 GraspSplats 中的显式和优化几何足够支持（1）实时抓取采样和（2）动态和关节状物体操作与点跟踪器。在 Franka 机器人上进行广泛的实验后，我们证明了 GraspSplats 在各种任务设置中显著优于现有方法。特别是，GraspSplats 超越了基于 NeRF 的方法（如 F3RM 和 LERF-TOGO）和 2D 检测方法。
URL
https://arxiv.org/abs/2409.02084
