Abstract
Within colorectal cancer diagnostics, conventional colonoscopy techniques face critical limitations, including a limited field of view and a lack of depth information, which can impede the detection of precancerous lesions. Current methods struggle to provide comprehensive and accurate 3D reconstructions of the colonic surface which can help minimize the missing regions and reinspection for pre-cancerous polyps. Addressing this, we introduce 'Gaussian Pancakes', a method that leverages 3D Gaussian Splatting (3D GS) combined with a Recurrent Neural Network-based Simultaneous Localization and Mapping (RNNSLAM) system. By introducing geometric and depth regularization into the 3D GS framework, our approach ensures more accurate alignment of Gaussians with the colon surface, resulting in smoother 3D reconstructions with novel viewing of detailed textures and structures. Evaluations across three diverse datasets show that Gaussian Pancakes enhances novel view synthesis quality, surpassing current leading methods with a 18% boost in PSNR and a 16% improvement in SSIM. It also delivers over 100X faster rendering and more than 10X shorter training times, making it a practical tool for real-time applications. Hence, this holds promise for achieving clinical translation for better detection and diagnosis of colorectal cancer.
Abstract (translated)
在直肠癌诊断中，传统的结肠镜检查技术面临着关键的限制，包括视野有限和深度信息缺乏，这可能阻碍了癌前病变的检测。目前的 methods 很难提供完整的和准确的 3D 结肠表面重建，这可以帮助最小化遗漏区域和重新评估癌前结肠癌。为了解决这个问题，我们引入了“Gaussian Pancakes”方法，这是一种利用 3D Gaussian Splatting（3D GS）与基于循环神经网络的同时定位与映射（RNNSLAM）系统相结合的方法。通过将几何和深度规范化到 3D GS 框架中，我们的方法确保了 Gaussian 与结肠表面的更准确对齐，从而实现了更平滑的 3D 重建，并重新观察到了详细纹理和结构的全新视角。在三个不同的数据集上的评估显示，Gaussian Pancakes 提高了新颖视角合成质量，超过现有领先方法，PSNR 提高了 18%，SSIM 提高了 16%。它还实现了超过 100X 的快速渲染和超过 10X 的训练时间，使得它成为实时应用的实用工具。因此，这有望为实现临床转化和改进直肠癌的检测和诊断带来希望。
URL
https://arxiv.org/abs/2404.06128
