Abstract
Incrementally recovering real-sized 3D geometry from a pose-free RGB stream is a challenging task in 3D reconstruction, requiring minimal assumptions on input data. Existing methods can be broadly categorized into end-to-end and visual SLAM-based approaches, both of which either struggle with long sequences or depend on slow test-time optimization and depth sensors. To address this, we first integrate a depth estimator into an RGB-D SLAM system, but this approach is hindered by inaccurate geometric details in predicted depth. Through further investigation, we find that 3D Gaussian mapping can effectively solve this problem. Building on this, we propose an online 3D reconstruction method using 3D Gaussian-based SLAM, combined with a feed-forward recurrent prediction module to directly infer camera pose from optical flow. This approach replaces slow test-time optimization with fast network inference, significantly improving tracking speed. Additionally, we introduce a local graph rendering technique to enhance robustness in feed-forward pose prediction. Experimental results on the Replica and TUM-RGBD datasets, along with a real-world deployment demonstration, show that our method achieves performance on par with the state-of-the-art SplaTAM, while reducing tracking time by more than 90\%.
Abstract (translated)
从无姿态RGB流中逐步恢复真实的3D几何形状是3D重建中的一个挑战性任务，该任务需要对输入数据做出最少的假设。现有的方法主要可以分为端到端和基于视觉SLAM（Simultaneous Localization and Mapping）的方法，这些方法要么难以处理长序列的问题，要么依赖于耗时的测试时间优化以及深度传感器。 为了解决这些问题，我们首先将一个深度估计器整合到了RGB-D SLAM系统中。然而这种方法由于预测出的深度不够准确导致几何细节不精确而受阻。通过进一步研究，我们发现基于3D高斯映射的方法可以有效解决这一问题。在此基础上，我们提出了一种使用基于3D高斯映射SLAM结合前馈递归预测模块直接从光流中推断相机姿态的在线3D重建方法。这种方法将耗时的测试时间优化替换为快速网络推理，显著提高了跟踪速度。此外，我们还引入了局部图渲染技术来增强前馈姿态预测的鲁棒性。 在Replica和TUM-RGBD数据集上的实验结果以及一个真实的部署演示表明，我们的方法与当前最先进的SplatMap方法性能相当，同时将跟踪时间减少了超过90%。
URL
https://arxiv.org/abs/2508.04597
