Abstract
3D instance segmentation is crucial for understanding complex 3D environments, yet fully supervised methods require dense point-level annotations, resulting in substantial annotation costs and labor overhead. To mitigate this, box-level annotations have been explored as a weaker but more scalable form of supervision. However, box annotations inherently introduce ambiguity in overlapping regions, making accurate point-to-instance assignment challenging. Recent methods address this ambiguity by generating pseudo-masks through training a dedicated pseudo-labeler in an additional training stage. However, such two-stage pipelines often increase overall training time and complexity, hinder end-to-end optimization. To overcome these challenges, we propose BEEP3D-Box-supervised End-to-End Pseudo-mask generation for 3D instance segmentation. BEEP3D adopts a student-teacher framework, where the teacher model serves as a pseudo-labeler and is updated by the student model via an Exponential Moving Average. To better guide the teacher model to generate precise pseudo-masks, we introduce an instance center-based query refinement that enhances position query localization and leverages features near instance centers. Additionally, we design two novel losses-query consistency loss and masked feature consistency loss-to align semantic and geometric signals between predictions and pseudo-masks. Extensive experiments on ScanNetV2 and S3DIS datasets demonstrate that BEEP3D achieves competitive or superior performance compared to state-of-the-art weakly supervised methods while remaining computationally efficient.
Abstract (translated)
三维实例分割对于理解复杂的三维环境至关重要，然而完全监督的方法需要密集的点级别标注，这导致了大量的注释成本和劳动力投入。为了缓解这一问题，人们探索了使用箱级标注作为更弱但更具可扩展性的监督形式。然而，边界框标注在重叠区域内在固有地引入了模糊性，使得准确的点到实例分配变得具有挑战性。近期的方法通过训练一个专门的伪标签生成器来解决这种模糊性，但这通常需要额外的一个训练阶段，并且会产生伪掩码。这样的两阶段管道往往会增加整体训练时间和复杂度，阻碍端到端优化。 为了解决这些挑战，我们提出了BEEP3D——一种基于边界框监督的三维实例分割中的端到端伪掩码生成方法。BEEP3D采用了一种师生框架，在该框架中，教师模型充当伪标签生成器，并通过指数移动平均法由学生模型进行更新。为了更好地指导教师模型生成精确的伪掩码，我们引入了一个基于实例中心的查询细化机制，增强了位置查询的定位能力，并利用了靠近实例中心的特征信息。此外，我们设计了两种新颖的损失函数——查询一致性损失和掩码特征一致性损失，以便在预测与伪掩码之间对齐语义和几何信号。 在ScanNetV2和S3DIS数据集上的广泛实验表明，BEEP3D相比现有的弱监督方法取得了具有竞争力或更优的性能，并且保持了计算效率。
URL
https://arxiv.org/abs/2510.12182
