Abstract
Detecting 3D keypoints with semantic consistency is widely used in many scenarios such as pose estimation, shape registration and robotics. Currently, most unsupervised 3D keypoint detection methods focus on the rigid-body objects. However, when faced with deformable objects, the keypoints they identify do not preserve semantic consistency well. In this paper, we introduce an innovative unsupervised keypoint detector Key-Grid for both the rigid-body and deformable objects, which is an autoencoder framework. The encoder predicts keypoints and the decoder utilizes the generated keypoints to reconstruct the objects. Unlike previous work, we leverage the identified keypoint in formation to form a 3D grid feature heatmap called grid heatmap, which is used in the decoder section. Grid heatmap is a novel concept that represents the latent variables for grid points sampled uniformly in the 3D cubic space, where these variables are the shortest distance between the grid points and the skeleton connected by keypoint pairs. Meanwhile, we incorporate the information from each layer of the encoder into the decoder section. We conduct an extensive evaluation of Key-Grid on a list of benchmark datasets. Key-Grid achieves the state-of-the-art performance on the semantic consistency and position accuracy of keypoints. Moreover, we demonstrate the robustness of Key-Grid to noise and downsampling. In addition, we achieve SE-(3) invariance of keypoints though generalizing Key-Grid to a SE(3)-invariant backbone.
Abstract (translated)
检测3D关键点与语义一致性是许多应用场景（如姿态估计、形状配准和机器人技术）中广泛使用的。目前，大多数无监督3D关键点检测方法都关注于刚体物体。然而，面对变形物体，它们确定的关键点在语义上并不保持一致。在本文中，我们提出了一种创新的无监督关键点检测器Key-Grid，适用于刚体和变形物体，是一种自动编码器框架。编码器预测关键点，解码器利用生成的关键点重构物体。与之前的工作不同，我们利用已识别的关键点形成一个3D立方空间中采样均匀的网格点特征热图，即网格热图，用于解码器部分。网格热图是一种新颖的概念，它表示在3D立方空间中，网格点与通过关键点对齐的骨架之间的最短距离。同时，我们将编码器每一层的有关信息融入解码器部分。我们在一系列基准数据集上对Key-Grid进行广泛评估。Key-Grid在关键点的语义一致性和位置精度上实现了最先进的性能。此外，我们还证明了Key-Grid对噪声和下采样具有鲁棒性。此外，通过将Key-Grid扩展到SE(3)-不变的骨干网络，我们实现了关键点的SE(3)不变性。
URL
https://arxiv.org/abs/2410.02237
