Abstract
Complementary to prevalent LiDAR and camera systems, millimeter-wave (mmWave) radar is robust to adverse weather conditions like fog, rainstorms, and blizzards but offers sparse point clouds. Current techniques enhance the point cloud by the supervision of LiDAR's data. However, high-performance LiDAR is notably expensive and is not commonly available on vehicles. This paper presents mmEMP, a supervised learning approach that enhances radar point clouds using a low-cost camera and an inertial measurement unit (IMU), enabling crowdsourcing training data from commercial vehicles. Bringing the visual-inertial (VI) supervision is challenging due to the spatial agnostic of dynamic objects. Moreover, spurious radar points from the curse of RF multipath make robots misunderstand the scene. mmEMP first devises a dynamic 3D reconstruction algorithm that restores the 3D positions of dynamic features. Then, we design a neural network that densifies radar data and eliminates spurious radar points. We build a new dataset in the real world. Extensive experiments show that mmEMP achieves competitive performance compared with the SOTA approach training by LiDAR's data. In addition, we use the enhanced point cloud to perform object detection, localization, and mapping to demonstrate mmEMP's effectiveness.
Abstract (translated)
作为当前普遍的LiDAR和相机系统功能的补充，毫米波（mmWave）雷达对逆天气条件如雾、雷暴和暴风雪等具有很强的鲁棒性，但输出点云稀疏。目前的方法通过LiDAR数据的监督来增强点云。然而，高性能LiDAR价格昂贵，在车辆上并不常见。本文提出了一种低成本相机和惯性测量单元（IMU）协同工作，通过自适应波束形成技术增强雷达点云，实现从商用车辆的大众培训数据。由于动态对象的局域性，视觉-惯性（VI）监督带来挑战。此外，来自RF多径污染的伪雷达点使机器人误解场景。mmEMP首先设计了一个动态3D重建算法，恢复了动态特征的3D位置。然后，我们设计了一个神经网络，通过增加雷达数据密度和消除伪雷达点来加强雷达数据。我们在现实世界中构建了一个新数据集。大量实验证明，与通过LiDAR数据训练的当前最佳方法相比，mmEMP具有竞争力的性能。此外，我们使用增强后的点云进行目标检测、定位和映射，以展示mmEMP的有效性。
URL
https://arxiv.org/abs/2404.17229
