Abstract
Monocular 3D object detection is a promising yet ill-posed task for autonomous vehicles due to the lack of accurate depth information. Cross-modality knowledge distillation could effectively transfer depth information from LiDAR to image-based network. However, modality gap between image and LiDAR seriously limits its accuracy. In this paper, we systematically investigate the negative transfer problem induced by modality gap in cross-modality distillation for the first time, including not only the architecture inconsistency issue but more importantly the feature overfitting issue. We propose a selective learning approach named MonoSTL to overcome these issues, which encourages positive transfer of depth information from LiDAR while alleviates the negative transfer on image-based network. On the one hand, we utilize similar architectures to ensure spatial alignment of features between image-based and LiDAR-based networks. On the other hand, we develop two novel distillation modules, namely Depth-Aware Selective Feature Distillation (DASFD) and Depth-Aware Selective Relation Distillation (DASRD), which selectively learn positive features and relationships of objects by integrating depth uncertainty into feature and relation distillations, respectively. Our approach can be seamlessly integrated into various CNN-based and DETR-based models, where we take three recent models on KITTI and a recent model on NuScenes for validation. Extensive experiments show that our approach considerably improves the accuracy of the base models and thereby achieves the best accuracy compared with all recently released SOTA models.
Abstract (translated)
单目3D物体检测对于自动驾驶汽车来说是一个有前景但又难以处理的任务，因为缺乏准确的深度信息。跨模态知识蒸馏可以有效地将来自激光雷达（LiDAR）的深度信息转移到基于图像的网络中。然而，由于图像和激光雷达之间的模态差距，这一方法严重限制了其准确性。在这篇论文中，我们首次系统地研究了由跨模态蒸馏中的模态差距所引起的负面迁移问题，包括架构不一致的问题以及更重要的特征过拟合问题。为了克服这些问题，我们提出了一种名为MonoSTL的选择性学习方法，该方法鼓励从激光雷达向基于图像的网络正向传输深度信息的同时缓解负向转移。 一方面，我们使用相似的架构以确保基于图像和基于LiDAR网络之间的空间对齐。另一方面，我们开发了两个新颖的蒸馏模块：深度感知选择性特征蒸馏（Depth-Aware Selective Feature Distillation, DASFD）和深度感知选择性关系蒸馏（Depth-Aware Selective Relation Distillation, DASRD）。这些模块通过将深度不确定性分别整合到特征蒸馏和关系蒸馏中，有选择地学习物体的正向特征和关系。 我们的方法可以无缝集成到各种基于CNN和DETR的模型中，在KITTI数据集上的三种最近模型以及NuScenes数据集上的一种最近模型进行了验证。广泛的实验表明，相较于所有最近发布的最先进（SOTA）模型，我们提出的这种方法显著提高了基础模型的准确性，并实现了最佳精度。
URL
https://arxiv.org/abs/2603.07464
