Abstract
Until open-world foundation models match the performance of specialized approaches, the effectiveness of deep learning models remains heavily dependent on dataset availability. Training data must align not only with the target object categories but also with the sensor characteristics and modalities. To bridge the gap between available datasets and deployment domains, domain adaptation strategies are widely used. In this work, we propose a novel approach to transferring sensor-specific knowledge from an image dataset to LiDAR, an entirely different sensing domain. Our method XD-MAP leverages detections from a neural network on camera images to create a semantic parametric map. The map elements are modeled to produce pseudo labels in the target domain without any manual annotation effort. Unlike previous domain transfer approaches, our method does not require direct overlap between sensors and enables extending the angular perception range from a front-view camera to a full 360 view. On our large-scale road feature dataset, XD-MAP outperforms single shot baseline approaches by +19.5 mIoU for 2D semantic segmentation, +19.5 PQth for 2D panoptic segmentation, and +32.3 mIoU in 3D semantic segmentation. The results demonstrate the effectiveness of our approach achieving strong performance on LiDAR data without any manual labeling.
Abstract (translated)
直到开放世界的基础模型能够达到专业化方法的性能，深度学习模型的有效性仍然很大程度上依赖于数据集的可用性。训练数据不仅要与目标对象类别对齐，还要与传感器特性和模式一致。为了弥合现有数据集和部署领域之间的差距，域适应策略被广泛应用。在这项工作中，我们提出了一种将特定传感器的知识从图像数据集中转移到LiDAR（一种完全不同的传感领域）的新型方法。我们的方法XD-MAP利用神经网络在摄像头图像上的检测结果创建语义参数化地图。该方法的地图元素可以生成目标域中的伪标签，而无需任何人工标注工作。与以往的域转换方法不同，我们的方法不需要传感器之间的直接重叠，并且能够将前视摄像头的角度感知范围扩展到360度全方位视角。在我们大规模的道路特征数据集上，XD-MAP在二维语义分割方面比单次基准方法高出+19.5 mIoU，在二维全景分割方面高出+19.5 PQth，在三维语义分割方面高出+32.3 mIoU。实验结果证明了我们的方法在不进行任何人工标注的情况下，对于LiDAR数据能够实现强大的性能。
URL
https://arxiv.org/abs/2601.14477
