Abstract
Visual SLAM is particularly challenging in environments affected by noise, varying lighting conditions, and darkness. Learning-based optical flow algorithms can leverage multiple modalities to address these challenges, but traditional optical flow-based visual SLAM approaches often require significant computational this http URL overcome this limitation, we propose FMF-SLAM, an efficient multimodal fusion SLAM method that utilizes fast Fourier transform (FFT) to enhance the algorithm efficiency. Specifically, we introduce a novel Fourier-based self-attention and cross-attention mechanism to extract features from RGB and depth signals. We further enhance the interaction of multimodal features by incorporating multi-scale knowledge distillation across modalities. We also demonstrate the practical feasibility of FMF-SLAM in real-world scenarios with real time performance by integrating it with a security robot by fusing with a global positioning module GNSS-RTK and global Bundle Adjustment. Our approach is validated using video sequences from TUM, TartanAir, and our real-world datasets, showcasing state-of-the-art performance under noisy, varying lighting, and dark this http URL code and datasets are available at this https URL.
Abstract (translated)
视觉SLAM在受噪声影响、光照变化以及黑暗环境下的表现特别具有挑战性。基于学习的光流算法可以通过利用多模态信息来应对这些挑战，但传统的基于光流的视觉SLAM方法往往需要大量的计算资源才能克服这一限制。为了克服这个局限性，我们提出了FMF-SLAM（快速傅里叶变换多模态融合SLAM），这是一种高效的多模态融合定位和建图算法，它利用快速傅里叶变换来提高算法效率。具体来说，我们引入了一种新的基于傅里叶的自注意力机制以及跨模态注意机制，用于从RGB信号和深度信号中提取特征。此外，通过在各尺度上进行跨模态知识蒸馏，我们也加强了多模态特征之间的交互作用。 为了展示FMF-SLAM在实际场景中的可行性并保持实时性能，我们将其与一个安防机器人集成，并融合了全球定位模块GNSS-RTK和全局捆绑调整。我们的方法通过使用TUM、TartanAir以及我们自己的真实数据集进行了验证，在嘈杂环境、光照变化大及黑暗环境下均表现出领先于现有技术的性能。 有关代码和数据集可在此网址获取：[请提供具体的URL]（假设原文本中的 "this https URL" 应为实际链接地址）。
URL
https://arxiv.org/abs/2506.18204
