Abstract
Instance segmentation of point clouds is a crucial task in 3D field with numerous applications that involve localizing and segmenting objects in a scene. However, achieving satisfactory results requires a large number of manual annotations, which is a time-consuming and expensive process. To alleviate dependency on annotations, we propose a method, called FreePoint, for underexplored unsupervised class-agnostic instance segmentation on point clouds. In detail, we represent the point features by combining coordinates, colors, normals, and self-supervised deep features. Based on the point features, we perform a multicut algorithm to segment point clouds into coarse instance masks as pseudo labels, which are used to train a point cloud instance segmentation model. To alleviate the inaccuracy of coarse masks during training, we propose a weakly-supervised training strategy and corresponding loss. Our work can also serve as an unsupervised pre-training pretext for supervised semantic instance segmentation with limited annotations. For class-agnostic instance segmentation on point clouds, FreePoint largely fills the gap with its fully-supervised counterpart based on the state-of-the-art instance segmentation model Mask3D and even surpasses some previous fully-supervised methods. When serving as a pretext task and fine-tuning on S3DIS, FreePoint outperforms training from scratch by 5.8% AP with only 10% mask annotations.
Abstract (translated)
点云实例分割是三维领域中一个重要的任务，有许多应用涉及在场景中定位和分割物体。然而，要实现满意的结果需要大量的手动标注，这是一个耗时且昂贵的过程。为了减轻依赖标注的情况，我们提出了一种方法，称为FreePoint，用于未受重视的点云 unsupervised 类特异性实例分割。具体来说，我们使用坐标、颜色、正弦和自监督的深度特征来代表点特征。基于点特征，我们执行多切算法将点云分割成粗实例 masks 作为伪标签，用于训练点云实例分割模型。为了减轻训练期间粗标签的不准确情况，我们提出了一种较弱的supervised 训练策略和相应的损失。我们的工作还可以作为 unsupervised 的前序训练目标，以有限标注的 supervised 语义实例分割。对于点云的类特异性实例分割，FreePoint 在很大程度上填补基于先进的实例分割模型Mask3D的未受重视的 gap 并与一些以前的完全监督方法相比。当作为前序训练目标和在S3DIS上进行 fine-tuning时，FreePoint 通过仅10%的 mask 标注比从零开始训练取得了5.8%的AP优势。
URL
https://arxiv.org/abs/2305.06973
