Abstract
Efficient three-dimensional reconstruction and real-time visualization are critical in surgical scenarios such as endoscopy. In recent years, 3D Gaussian Splatting (3DGS) has demonstrated remarkable performance in efficient 3D reconstruction and rendering. Most 3DGS-based Simultaneous Localization and Mapping (SLAM) methods only rely on the appearance constraints for optimizing both 3DGS and camera poses. However, in endoscopic scenarios, the challenges include photometric inconsistencies caused by non-Lambertian surfaces and dynamic motion from breathing affects the performance of SLAM systems. To address these issues, we additionally introduce optical flow loss as a geometric constraint, which effectively constrains both the 3D structure of the scene and the camera motion. Furthermore, we propose a depth regularisation strategy to mitigate the problem of photometric inconsistencies and ensure the validity of 3DGS depth rendering in endoscopic scenes. In addition, to improve scene representation in the SLAM system, we improve the 3DGS refinement strategy by focusing on viewpoints corresponding to Keyframes with suboptimal rendering quality frames, achieving better rendering results. Extensive experiments on the C3VD static dataset and the StereoMIS dynamic dataset demonstrate that our method outperforms existing state-of-the-art methods in novel view synthesis and pose estimation, exhibiting high performance in both static and dynamic surgical scenes. The source code will be publicly available upon paper acceptance.
Abstract (translated)
在手术场景（如内窥镜）中，高效的三维重建和实时可视化至关重要。近年来，3D高斯点阵法 (3DGaussian Splatting, 3DGS) 在高效三维重建和渲染方面表现出了卓越的性能。大多数基于3DGS的同时定位与地图构建(Simultaneous Localization and Mapping, SLAM)方法仅依赖于外观约束来优化3DGS以及相机姿态。然而，在内窥镜场景中，非朗伯表面引起的光度不一致性和呼吸导致的动态运动会对SLAM系统的性能产生挑战。 为了解决这些问题，我们额外引入了光学流损失作为几何约束，该约束有效限制了场景中的三维结构和摄像机移动。此外，我们提出了一种深度正则化策略来缓解光度不一致性的问题，并确保3DGS在内窥镜场景中渲染的深度的有效性。为了提高SLAM系统中的场景表示，我们改进了3DGS优化策略，重点关注关键帧对应的子优图像视点，从而获得更好的渲染效果。 在C3VD静态数据集和StereoMIS动态数据集上的广泛实验表明，我们的方法在新视角合成和姿态估计方面超越现有的最先进的技术，在静态和动态手术场景中都表现出色。论文被接受后，源代码将公开发布。
URL
https://arxiv.org/abs/2506.21420
