Abstract
Monocular 3D object detection is an essential task in computer vision, and it has several applications in robotics and virtual reality. However, 3D object detectors are typically trained in a fully supervised way, relying extensively on 3D labeled data, which is labor-intensive and costly to annotate. This work focuses on weakly-supervised 3D detection to reduce data needs using a monocular method that leverages a singlecamera system over expensive LiDAR sensors or multi-camera setups. We propose a general model Weak Cube R-CNN, which can predict objects in 3D at inference time, requiring only 2D box annotations for training by exploiting the relationship between 2D projections of 3D cubes. Our proposed method utilizes pre-trained frozen foundation 2D models to estimate depth and orientation information on a training set. We use these estimated values as pseudo-ground truths during training. We design loss functions that avoid 3D labels by incorporating information from the external models into the loss. In this way, we aim to implicitly transfer knowledge from these large foundation 2D models without having access to 3D bounding box annotations. Experimental results on the SUN RGB-D dataset show increased performance in accuracy compared to an annotation time equalized Cube R-CNN baseline. While not precise for centimetre-level measurements, this method provides a strong foundation for further research.
Abstract (translated)
单目三维物体检测是计算机视觉中的一个核心任务，在机器人技术和虚拟现实领域中有多种应用。然而，通常情况下，3D对象检测器需要在完全监督的环境下进行训练，并且严重依赖于标注详细的3D数据，这种标注过程既费时又昂贵。这项工作专注于弱监督下的3D检测，以减少对大量标注数据的需求，采用单目方法利用单一摄像头系统而不是使用成本高昂的激光雷达传感器或多相机设置。 我们提出了一种名为“弱Cube R-CNN”的通用模型，该模型能够在推理阶段预测三维对象的位置，并且只需2D框注释进行训练。通过利用3D立方体在二维平面上投影的关系，我们的方法可以实现这一点。此外，我们使用预训练的冻结基础2D模型来估计数据集上的深度和方向信息，并将这些估计值作为伪地面实况用于训练阶段。 为了不依赖于3D标签，在损失函数的设计中，我们将外部模型的信息整合进损失计算之中，从而在没有直接访问3D边界框注释的情况下实现从大型基础2D模型到本模型的知识转移。实验结果表明，在SUN RGB-D数据集上，与等时标注的Cube R-CNN基线相比，我们的方法在精度方面表现出色。尽管该方法对于毫米级测量不够精确，但它为进一步的研究提供了坚实的基础。
URL
https://arxiv.org/abs/2504.13297
