Abstract
In the fields of photogrammetry, computer vision and computer graphics, the task of neural 3D scene reconstruction has led to the exploration of various techniques. Among these, 3D Gaussian Splatting stands out for its explicit representation of scenes using 3D Gaussians, making it appealing for tasks like 3D point cloud extraction and surface reconstruction. Motivated by its potential, we address the domain of 3D scene reconstruction, aiming to leverage the capabilities of the Microsoft HoloLens 2 for instant 3D Gaussian Splatting. We present HoloGS, a novel workflow utilizing HoloLens sensor data, which bypasses the need for pre-processing steps like Structure from Motion by instantly accessing the required input data i.e. the images, camera poses and the point cloud from depth sensing. We provide comprehensive investigations, including the training process and the rendering quality, assessed through the Peak Signal-to-Noise Ratio, and the geometric 3D accuracy of the densified point cloud from Gaussian centers, measured by Chamfer Distance. We evaluate our approach on two self-captured scenes: An outdoor scene of a cultural heritage statue and an indoor scene of a fine-structured plant. Our results show that the HoloLens data, including RGB images, corresponding camera poses, and depth sensing based point clouds to initialize the Gaussians, are suitable as input for 3D Gaussian Splatting.
Abstract (translated)
在摄影测量、计算机视觉和计算机图形学领域，神经3D场景重建的任务促使我们探索各种技术。在这些技术中，3D高斯平铺因使用3D高斯形式的场景表示而脱颖而出，这使得它对诸如3D点云提取和表面重建等任务具有吸引力。为了利用其潜力，我们转向3D场景重建领域，旨在利用微软HoloLens 2的即时3D高斯平铺功能。我们介绍了HoloGS，一种利用HoloLens传感器数据的新工作流程，无需进行预处理步骤，即可直接访问所需输入数据，即深度感測的图像、摄影机姿态和点云。我们提供了全面的调查，包括训练过程和渲染质量，通过Peak信号-噪声比进行评估，以及通过Chamfer距离测量来自高斯中心的密度点云的3D几何准确性。我们对我们的方法在两个自 capture场景进行了评估：文化遗产雕像的户外场景和精细结构植物的室内场景。我们的结果表明，HoloLens数据，包括RGB图像、相应的相机姿态和基于深度感測的点云，作为3D高斯平铺的输入是合适的。
URL
https://arxiv.org/abs/2405.02005
