Abstract
Bounding box supervision has gained considerable attention in weakly supervised 3D instance segmentation. While this approach alleviates the need for extensive point-level annotations, obtaining accurate bounding boxes in practical applications remains challenging. To this end, we explore the inaccurate bounding box, named sketchy bounding box, which is imitated through perturbing ground truth bounding box by adding scaling, translation, and rotation. In this paper, we propose Sketchy-3DIS, a novel weakly 3D instance segmentation framework, which jointly learns pseudo labeler and segmentator to improve the performance under the sketchy bounding-box supervisions. Specifically, we first propose an adaptive box-to-point pseudo labeler that adaptively learns to assign points located in the overlapped parts between two sketchy bounding boxes to the correct instance, resulting in compact and pure pseudo instance labels. Then, we present a coarse-to-fine instance segmentator that first predicts coarse instances from the entire point cloud and then learns fine instances based on the region of coarse instances. Finally, by using the pseudo instance labels to supervise the instance segmentator, we can gradually generate high-quality instances through joint training. Extensive experiments show that our method achieves state-of-the-art performance on both the ScanNetV2 and S3DIS benchmarks, and even outperforms several fully supervised methods using sketchy bounding boxes. Code is available at this https URL.
Abstract (translated)
边界框监督在弱监督下的三维实例分割中受到了广泛的关注。尽管这种方法减轻了对大量点级别标注的需求，但在实际应用中获取准确的边界框仍然具有挑战性。为此，我们探索了一种不精确的边界框，称为草图边界框，它是通过对地面真实边界框添加缩放、平移和旋转扰动来模仿得到的。 在本文中，我们提出了一种新颖的弱监督三维实例分割框架——Sketchy-3DIS，该框架通过联合学习伪标签生成器（pseudo labeler）和分割器（segmentator），以改善草图边界框指导下的性能。具体而言，首先我们提出了一个自适应的框到点的伪标签生成器，它能够自适应地将位于两个不精确边界框重叠部分中的点分配给正确的实例，从而产生紧凑且纯净的伪实例标签。然后，我们提出了一种由粗到精的实例分割器，该分割器首先从整个点云中预测粗略实例，然后再基于这些粗略实例区域学习精细实例。最后，通过使用伪实例标签监督实例分割器，并进行联合训练，可以逐步生成高质量的实例。 大量的实验表明，我们的方法在ScanNetV2和S3DIS基准测试上取得了最先进的性能，甚至超过了几个采用草图边界框的完全监督的方法。代码可在提供的链接中获取。
URL
https://arxiv.org/abs/2505.16399
