Abstract
Existing state-of-the-art 3D point clouds understanding methods only perform well in a fully supervised manner. To the best of our knowledge, there exists no unified framework which simultaneously solves the downstream high-level understanding tasks, especially when labels are extremely limited. This work presents a general and simple framework to tackle point clouds understanding when labels are limited. We propose a novel unsupervised region expansion based clustering method for generating clusters. More importantly, we innovatively propose to learn to merge the over-divided clusters based on the local low-level geometric property similarities and the learned high-level feature similarities supervised by weak labels. Hence, the true weak labels guide pseudo labels merging taking both geometric and semantic feature correlations into consideration. Finally, the self-supervised reconstruction and data augmentation optimization modules are proposed to guide the propagation of labels among semantically similar points within a scene. Experimental Results demonstrate that our framework has the best performance among the three most important weakly supervised point clouds understanding tasks including semantic segmentation, instance segmentation, and object detection even when limited points are labeled, under the data-efficient settings for the large-scale 3D semantic scene parsing. The developed techniques have postentials to be applied to downstream tasks for better representations in robotic manipulation and robotic autonomous navigation. Codes and models are publicly available at: this https URL.
Abstract (translated)
目前最先进的3D点云理解方法仅在完全监督的情况下表现良好。据我们所知，还没有一个统一框架能够同时解决下游的高级理解任务，特别是在标签非常有限的情况下。本文提出了一种通用的且简单的框架来解决标签有限时的点云理解问题。我们提出了一种新颖的自监督聚类方法生成聚类。更重要的是，我们创新地提出了一种基于局部低级几何性质相似度和由弱标签学习到的较高级特征相似度的聚类方法，从而使真正的弱标签引导伪标签的合并。因此，在考虑几何和语义特征相关性的情况下，指导场景内同义点之间的标签传播。最后，我们提出了自监督重构和数据增强优化模块，以引导大规模3D语义场景解析中标签的传播。实验结果表明，在考虑有限点标签的情况下，我们的框架在包括语义分割、实例分割和目标检测的三个最重要的弱监督点云理解任务中具有最佳性能。这些开发的技术具有将应用于机器人操作和机器人自主导航下游任务的潜在优势。代码和模型公开可用，此处链接：https://this URL。
URL
https://arxiv.org/abs/2312.02208
