Abstract
Monocular 3D object detection is a fundamental yet challenging task in 3D scene understanding. Existing approaches heavily depend on supervised learning with extensive 3D annotations, which are often acquired from LiDAR point clouds through labor-intensive labeling processes. To tackle this problem, we propose VSRD++, a novel weakly supervised framework for monocular 3D object detection that eliminates the reliance on 3D annotations and leverages neural-field-based volumetric rendering with weak 2D supervision. VSRD++ consists of a two-stage pipeline: multi-view 3D autolabeling and subsequent monocular 3D detector training. In the multi-view autolabeling stage, object surfaces are represented as signed distance fields (SDFs) and rendered as instance masks via the proposed instance-aware volumetric silhouette rendering. To optimize 3D bounding boxes, we decompose each instance's SDF into a cuboid SDF and a residual distance field (RDF) that captures deviations from the cuboid. To address the geometry inconsistency commonly observed in volume rendering methods applied to dynamic objects, we model the dynamic objects by including velocity into bounding box attributes as well as assigning confidence to each pseudo-label. Moreover, we also employ a 3D attribute initialization module to initialize the dynamic bounding box parameters. In the monocular 3D object detection phase, the optimized 3D bounding boxes serve as pseudo labels for training monocular 3D object detectors. Extensive experiments on the KITTI-360 dataset demonstrate that VSRD++ significantly outperforms existing weakly supervised approaches for monocular 3D object detection on both static and dynamic scenes. Code is available at this https URL
Abstract (translated)
单目三维物体检测是三维场景理解中的一个基本但具有挑战性的任务。现有的方法依赖于监督学习，需要大量的三维标注数据，这些标注通常通过耗时的激光雷达点云标记过程获得。为了解决这个问题，我们提出了一种名为VSRD++的新颖弱监督框架，用于单目三维物体检测，该框架消除了对3D注释的依赖，并利用基于神经场的体渲染技术，仅使用2D弱标注进行训练。 VSRD++包含一个两阶段管道：多视图3D自动标记和随后的单目3D检测器训练。在多视图自动标签阶段中，物体表面被表示为符号距离场（SDFs），并通过提出的实例感知体轮廓渲染方法将其渲染成实例掩模。为了优化三维边界框，我们将每个实例的SDF分解为一个立方体SDF和一个残差距离场（RDF），后者捕捉偏离立方体的距离偏差。 此外，针对体积渲染方法应用于动态对象时常见的几何不一致性问题，我们通过将速度纳入边界盒属性以及为每一个伪标签分配置信度来对动态物体进行建模。同时，我们也采用了3D属性初始化模块来初始化动态边框参数。 在单目三维物体检测阶段中，优化后的三维边界框被用作训练单目三维物体检测器的伪标签。在KITTI-360数据集上的广泛实验表明，VSRD++显著优于现有的弱监督方法，在静态和动态场景下的性能均领先于其他方法。 代码可以在提供的链接中获取（原文中的“this https URL”）。
URL
https://arxiv.org/abs/2512.01178
