Abstract
Large garages are ubiquitous yet intricate scenes in our daily lives, posing challenges characterized by monotonous colors, repetitive patterns, reflective surfaces, and transparent vehicle glass. Conventional Structure from Motion (SfM) methods for camera pose estimation and 3D reconstruction fail in these environments due to poor correspondence construction. To address these challenges, this paper introduces LetsGo, a LiDAR-assisted Gaussian splatting approach for large-scale garage modeling and rendering. We develop a handheld scanner, Polar, equipped with IMU, LiDAR, and a fisheye camera, to facilitate accurate LiDAR and image data scanning. With this Polar device, we present a GarageWorld dataset consisting of five expansive garage scenes with diverse geometric structures and will release the dataset to the community for further research. We demonstrate that the collected LiDAR point cloud by the Polar device enhances a suite of 3D Gaussian splatting algorithms for garage scene modeling and rendering. We also propose a novel depth regularizer for 3D Gaussian splatting algorithm training, effectively eliminating floating artifacts in rendered images, and a lightweight Level of Detail (LOD) Gaussian renderer for real-time viewing on web-based devices. Additionally, we explore a hybrid representation that combines the advantages of traditional mesh in depicting simple geometry and colors (e.g., walls and the ground) with modern 3D Gaussian representations capturing complex details and high-frequency textures. This strategy achieves an optimal balance between memory performance and rendering quality. Experimental results on our dataset, along with ScanNet++ and KITTI-360, demonstrate the superiority of our method in rendering quality and resource efficiency.
Abstract (translated)
大型车库在我们的日常生活中无处不在,但它们复杂的场景却鲜为人知。由于 poor correspondence construction,传统的相机姿态估计和3D重建方法在这些问题中失败。为了解决这些问题,本文介绍了Let'sGo,一种带有激光雷达的Gaussian插值方法,用于大型车库建模和渲染。我们开发了一种便携式扫描仪Polar,配备了惯性导航器、激光雷达和鱼眼相机,以帮助准确扫描 LiDAR 和图像数据。使用Polar设备,我们提出了一个由五个不同几何结构的宽敞车库场景组成的GarageWorld数据集,并将数据集发布给社区进行进一步研究。我们证明了Polar设备收集的LiDAR点云增强了用于车库场景建模和渲染的一组3D Gaussian插值算法。我们还提出了用于3D Gaussian插值算法训练的新型深度正则器,有效地消除了渲染图像中的浮动伪影,并开发了一个轻量级的实时级别细节(LOD)高斯渲染器,用于在基于web的设备上进行实时查看。此外,我们还探索了一种结合传统网格表示简单几何和颜色(例如墙壁和地面)与现代3D Gaussian表示捕捉复杂细节和高频纹理的混合表示策略。这种策略在内存性能和渲染质量之间实现了最优平衡。我们数据集上的实验结果,加上ScanNet++和KITTI-360,证明了我们在渲染质量和资源效率方面的优越性。
URL
https://arxiv.org/abs/2404.09748
