Abstract
The emerging trend in computer vision emphasizes developing universal models capable of simultaneously addressing multiple diverse tasks. Such universality typically requires joint training across multi-domain datasets to ensure effective generalization. However, monocular 3D object detection presents unique challenges in multi-domain training due to the scarcity of datasets annotated with accurate 3D ground-truth labels, especially beyond typical road-based autonomous driving contexts. To address this challenge, we introduce a novel weakly supervised framework leveraging pseudo-labels. Current pretrained models often struggle to accurately detect pedestrians in non-road environments due to inherent dataset biases. Unlike generalized image-based 2D object detection models, achieving similar generalization in monocular 3D detection remains largely unexplored. In this paper, we propose GATE3D, a novel framework designed specifically for generalized monocular 3D object detection via weak supervision. GATE3D effectively bridges domain gaps by employing consistency losses between 2D and 3D predictions. Remarkably, our model achieves competitive performance on the KITTI benchmark as well as on an indoor-office dataset collected by us to evaluate the generalization capabilities of our framework. Our results demonstrate that GATE3D significantly accelerates learning from limited annotated data through effective pre-training strategies, highlighting substantial potential for broader impacts in robotics, augmented reality, and virtual reality applications. Project page: this https URL
Abstract (translated)
计算机视觉领域的新趋势侧重于开发能够同时处理多种不同任务的通用模型。这种普遍性通常需要跨多域数据集进行联合训练，以确保有效的泛化能力。然而，在多域训练中，单目三维物体检测面临着独特挑战，尤其是在基于道路的自动驾驶场景之外，准确标注3D地面真实标签的数据集非常稀缺。为了解决这一难题，我们提出了一种新的弱监督框架，利用伪标签进行学习。 当前预训练模型在非道路环境中准确检测行人时往往遇到困难，这主要是由于数据集中固有的偏差所致。与泛化的基于图像的二维物体检测模型不同，在单目三维检测中实现类似的泛化能力仍然是一个未被充分探索的问题。本文提出了GATE3D框架，该框架专门针对通过弱监督进行泛化的单目三维物体检测设计。 GATE3D通过在2D和3D预测之间使用一致性损失来有效地弥合领域差距。值得注意的是，我们的模型不仅在KITTI基准测试中取得了竞争性的性能，在我们收集的用于评估我们框架泛化能力的室内办公室数据集上也表现出色。实验结果表明，GATE3D能够通过有效的预训练策略从有限标注的数据中快速学习，并展示了其在机器人、增强现实和虚拟现实应用中的广泛应用潜力。 项目主页：[请访问此链接获取更多信息](https://this.url.com)
URL
https://arxiv.org/abs/2504.11014
