Abstract
Transparent objects are ubiquitous in daily life, making their perception and robotics manipulation important. However, they present a major challenge due to their distinct refractive and reflective properties when it comes to accurately estimating the 6D pose. To solve this, we present ReFlow6D, a novel method for transparent object 6D pose estimation that harnesses the refractive-intermediate representation. Unlike conventional approaches, our method leverages a feature space impervious to changes in RGB image space and independent of depth information. Drawing inspiration from image matting, we model the deformation of the light path through transparent objects, yielding a unique object-specific intermediate representation guided by light refraction that is independent of the environment in which objects are observed. By integrating these intermediate features into the pose estimation network, we show that ReFlow6D achieves precise 6D pose estimation of transparent objects, using only RGB images as input. Our method further introduces a novel transparent object compositing loss, fostering the generation of superior refractive-intermediate features. Empirical evaluations show that our approach significantly outperforms state-of-the-art methods on TOD and Trans32K-6D datasets. Robot grasping experiments further demonstrate that ReFlow6D's pose estimation accuracy effectively translates to real-world robotics task. The source code is available at: this https URL and this https URL.
Abstract (translated)
透明物体在日常生活中随处可见，因此对其感知和机器人操控非常重要。然而，由于它们独特的折射和反射特性，在准确估计其6D姿态方面存在重大挑战。为了解决这一问题，我们提出了一种名为ReFlow6D的新方法，该方法利用了折射中间表示来估算透明对象的6D姿态。与传统方法不同的是，我们的方法利用了一个不受RGB图像空间变化影响且独立于深度信息的特征空间。受到图像抠图技术的启发，我们建模了光线穿过透明物体时路径的变形，从而生成了一种独特的、特定于每个对象的中间表示形式，这种表示完全不受观察环境的影响，并由光折射引导。通过将这些中间特征整合到姿态估计网络中，ReFlow6D展示了使用仅RGB图像作为输入即可实现精确估算透明物体6D姿态的能力。此外，我们的方法还引入了一种新颖的透明物体制作损失函数（transparent object compositing loss），以促进产生更优质的折射-中间特征。实证评估表明，在TOD和Trans32K-6D数据集上，我们所提出的方法显著优于现有的先进方法。机器人抓取实验进一步证明了ReFlow6D的姿势估计准确性可以有效应用于现实世界的机器人任务中。 源代码可在以下链接获取： - [链接1](this https URL) - [链接2](this https URL)
URL
https://arxiv.org/abs/2412.20830
