Abstract
Monocular 3D object detection poses a significant challenge in 3D scene understanding due to its inherently ill-posed nature in monocular depth estimation. Existing methods heavily rely on supervised learning using abundant 3D labels, typically obtained through expensive and labor-intensive annotation on LiDAR point clouds. To tackle this problem, we propose a novel weakly supervised 3D object detection framework named VSRD (Volumetric Silhouette Rendering for Detection) to train 3D object detectors without any 3D supervision but only weak 2D supervision. VSRD consists of multi-view 3D auto-labeling and subsequent training of monocular 3D object detectors using the pseudo labels generated in the auto-labeling stage. In the auto-labeling stage, we represent the surface of each instance as a signed distance field (SDF) and render its silhouette as an instance mask through our proposed instance-aware volumetric silhouette rendering. To directly optimize the 3D bounding boxes through rendering, we decompose the SDF of each instance into the SDF of a cuboid and the residual distance field (RDF) that represents the residual from the cuboid. This mechanism enables us to optimize the 3D bounding boxes in an end-to-end manner by comparing the rendered instance masks with the ground truth instance masks. The optimized 3D bounding boxes serve as effective training data for 3D object detection. We conduct extensive experiments on the KITTI-360 dataset, demonstrating that our method outperforms the existing weakly supervised 3D object detection methods. The code is available at this https URL.
Abstract (translated)
单目3D物体检测在3D场景理解中面临一个显著的挑战，因为其固有的不正确性在单目深度估计中。现有的方法在很大程度上依赖于使用丰富3D标签的监督学习，通常是通过在激光雷达点云上花费昂贵且费力的人工标注来获得的。为了解决这个问题，我们提出了一种新颖的弱监督3D物体检测框架，名为VSRD（体积轮廓渲染检测），以在没有3D监督的情况下训练3D物体检测器，但仅使用弱2D监督。VSRD包括多视角3D自动标注和自动标注阶段生成伪标签后训练单目3D物体检测器。在自动标注阶段，我们将每个实例的表面表示为一个有符号距离场（SDF），并通过我们提出的实例感知体积轮廓渲染将其轮廓渲染为实例掩码。为了通过渲染直接优化3D边界框，我们将每个实例的SDF分解为立方体的SDF和残差距离场（RDF），这使得我们能够通过比较渲染实例掩码与真实实例掩码来优化3D边界框。通过在KITTI-360数据集上进行广泛的实验，我们证明了我们的方法优于现有的弱监督3D物体检测方法。代码可在此处访问：https://www.kitti.org/data/
URL
https://arxiv.org/abs/2404.00149
