Paper Reading AI Learner

FreePoint: Unsupervised Point Cloud Instance Segmentation

2023-05-11 16:56:26
Zhikai Zhang, Jian Ding, Li Jiang, Dengxin Dai, Gui-Song Xia


Instance segmentation of point clouds is a crucial task in 3D field with numerous applications that involve localizing and segmenting objects in a scene. However, achieving satisfactory results requires a large number of manual annotations, which is a time-consuming and expensive process. To alleviate dependency on annotations, we propose a method, called FreePoint, for underexplored unsupervised class-agnostic instance segmentation on point clouds. In detail, we represent the point features by combining coordinates, colors, normals, and self-supervised deep features. Based on the point features, we perform a multicut algorithm to segment point clouds into coarse instance masks as pseudo labels, which are used to train a point cloud instance segmentation model. To alleviate the inaccuracy of coarse masks during training, we propose a weakly-supervised training strategy and corresponding loss. Our work can also serve as an unsupervised pre-training pretext for supervised semantic instance segmentation with limited annotations. For class-agnostic instance segmentation on point clouds, FreePoint largely fills the gap with its fully-supervised counterpart based on the state-of-the-art instance segmentation model Mask3D and even surpasses some previous fully-supervised methods. When serving as a pretext task and fine-tuning on S3DIS, FreePoint outperforms training from scratch by 5.8% AP with only 10% mask annotations.

Abstract (translated)

点云实例分割是三维领域中一个重要的任务,有许多应用涉及在场景中定位和分割物体。然而,要实现满意的结果需要大量的手动标注,这是一个耗时且昂贵的过程。为了减轻依赖标注的情况,我们提出了一种方法,称为FreePoint,用于未受重视的点云 unsupervised 类特异性实例分割。具体来说,我们使用坐标、颜色、正弦和自监督的深度特征来代表点特征。基于点特征,我们执行多切算法将点云分割成粗实例 masks 作为伪标签,用于训练点云实例分割模型。为了减轻训练期间粗标签的不准确情况,我们提出了一种较弱的supervised 训练策略和相应的损失。我们的工作还可以作为 unsupervised 的前序训练目标,以有限标注的 supervised 语义实例分割。对于点云的类特异性实例分割,FreePoint 在很大程度上填补基于先进的实例分割模型Mask3D的未受重视的 gap 并与一些以前的完全监督方法相比。当作为前序训练目标和在S3DIS上进行 fine-tuning时,FreePoint 通过仅10%的 mask 标注比从零开始训练取得了5.8%的AP优势。



3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot