Abstract
3D object pose estimation is a challenging task. Previous works always require thousands of object images with annotated poses for learning the 3D pose correspondence, which is laborious and time-consuming for labeling. In this paper, we propose to learn a category-level 3D object pose estimator without pose annotations. Instead of using manually annotated images, we leverage diffusion models (e.g., Zero-1-to-3) to generate a set of images under controlled pose differences and propose to learn our object pose estimator with those images. Directly using the original diffusion model leads to images with noisy poses and artifacts. To tackle this issue, firstly, we exploit an image encoder, which is learned from a specially designed contrastive pose learning, to filter the unreasonable details and extract image feature maps. Additionally, we propose a novel learning strategy that allows the model to learn object poses from those generated image sets without knowing the alignment of their canonical poses. Experimental results show that our method has the capability of category-level object pose estimation from a single shot setting (as pose definition), while significantly outperforming other state-of-the-art methods on the few-shot category-level object pose estimation benchmarks.
Abstract (translated)
3D物体姿态估计是一个具有挑战性的任务。以前的工作总是需要成千上万的带有注释姿态的物体图像才能学习3D姿态匹配，这费力且耗时。在本文中，我们提出了一种不需要姿态注释的类别级别3D物体姿态估计器。我们利用扩散模型（例如，零到一的扩散模型）在控制姿态差异的集合上生成一系列图像，并利用这些图像来学习我们的物体姿态估计器。直接使用原始扩散模型会导致带有噪音的姿态和伪影。为了解决这个问题，首先，我们利用专门设计的对比性姿态学习来提取图像特征，过滤不合理的细节。其次，我们提出了一种新的学习策略，使得模型能够从生成的图像集中学习物体姿态，而无需知道其全姿态的对齐。实验结果表明，我们的方法具有从单张照片设置中进行类别级别物体姿态估计的能力（作为姿态定义），同时在几个 shots的分类级别物体姿态估计基准上显著优于其他最先进的方法。
URL
https://arxiv.org/abs/2404.05626
