Abstract
LiDAR-camera extrinsic calibration (LCEC) is crucial for data fusion in intelligent vehicles. Offline, target-based approaches have long been the preferred choice in this field. However, they often demonstrate poor adaptability to real-world environments. This is largely because extrinsic parameters may change significantly due to moderate shocks or during extended operations in environments with vibrations. In contrast, online, target-free approaches provide greater adaptability yet typically lack robustness, primarily due to the challenges in cross-modal feature matching. Therefore, in this article, we unleash the full potential of large vision models (LVMs), which are emerging as a significant trend in the fields of computer vision and robotics, especially for embodied artificial intelligence, to achieve robust and accurate online, target-free LCEC across a variety of challenging scenarios. Our main contributions are threefold: we introduce a novel framework known as MIAS-LCEC, provide an open-source versatile calibration toolbox with an interactive visualization interface, and publish three real-world datasets captured from various indoor and outdoor environments. The cornerstone of our framework and toolbox is the cross-modal mask matching (C3M) algorithm, developed based on a state-of-the-art (SoTA) LVM and capable of generating sufficient and reliable matches. Extensive experiments conducted on these real-world datasets demonstrate the robustness of our approach and its superior performance compared to SoTA methods, particularly for the solid-state LiDARs with super-wide fields of view.
Abstract (translated)
LiDAR相机外差校准（LCEC）在智能汽车数据融合中至关重要。离线，目标导向的方法在领域中一直是首选。然而，它们通常在现实环境中表现不佳。这主要是因为外差参数可能会因中度冲击或环境振动而显著变化。相比之下，在线，目标无的方法提供更大的适应性，但通常缺乏鲁棒性，主要原因是跨模态特征匹配的挑战。因此，在本文中，我们发挥了大型视觉模型（LVMs）的全部潜力，这些模型在计算机视觉和机器人领域正成为一种趋势，特别是对于嵌入式人工智能，实现跨各种具有挑战性的场景的稳健且准确的在线，目标无LCEC。我们的主要贡献是三方面的：我们引入了一个名为MIAS-LCEC的新框架，提供了一个具有交互式可视化界面的开源多功能校准工具箱，并公开了从各种室内和室外环境捕获的三个真实世界数据集。我们框架和工具箱的基础是先进的基于SoTA LVM的跨模态掩码匹配（C3M）算法，能够生成充分且可靠的匹配。在这些真实世界数据集上进行的大量实验证明了我们方法的可行性和与SoTA方法的优越性能，特别是对于具有超宽视野的固体LiDAR。
URL
https://arxiv.org/abs/2404.18083
