Abstract
The use of Augmented Reality (AR) devices for surgical guidance has gained increasing traction in the medical field. Traditional registration methods often rely on external fiducial markers to achieve high accuracy and real-time performance. However, these markers introduce cumbersome calibration procedures and can be challenging to deploy in clinical settings. While commercial solutions have attempted real-time markerless tracking using the native RGB cameras of AR devices, their accuracy remains questionable for medical guidance, primarily due to occlusions and significant outliers between the live sensor data and the preoperative target anatomy point cloud derived from MRI or CT scans. In this work, we present a markerless framework that relies only on the depth sensor of AR devices and consists of two modules: a registration module for high-precision, outlier-robust target anatomy localization, and a tracking module for real-time pose estimation. The registration module integrates depth sensor error correction, a human-in-the-loop region filtering technique, and a robust global alignment with curvature-aware feature sampling, followed by local ICP refinement, for markerless alignment of preoperative models with patient anatomy. The tracking module employs a fast and robust registration algorithm that uses the initial pose from the registration module to estimate the target pose in real-time. We comprehensively evaluated the performance of both modules through simulation and real-world measurements. The results indicate that our markerless system achieves superior performance for registration and comparable performance for tracking to industrial solutions. The two-module design makes our system a one-stop solution for surgical procedures where the target anatomy moves or stays static during surgery.
Abstract (translated)
在医学领域，用于手术指导的增强现实（AR）设备的应用越来越受到重视。传统的注册方法通常依赖于外部标识物来实现高精度和实时性能，但这些标识物引入了繁琐的校准程序，并且在临床环境中部署起来颇具挑战性。尽管商用解决方案尝试使用AR设备内置的RGB摄像头进行无标记实时跟踪，但由于活体传感器数据与MRI或CT扫描获取的术前目标解剖点云之间存在遮挡和显著差异，这些方案的准确性仍不足以用于医学指导。 在这项工作中，我们提出了一种仅依赖于AR设备深度传感器的无标识物框架。该框架包括两个模块：一个是实现高精度、鲁棒异常值的目标解剖定位注册模块；另一个是实时姿态估计跟踪模块。注册模块集成了深度传感器误差校正技术、人机交互区域过滤技术和具有曲率感知特征采样的稳健全局对齐，随后进行局部ICP（迭代最近点）优化，以实现术前模型与患者解剖结构的无标记对准。跟踪模块采用了一种快速且稳健的注册算法，利用来自注册模块的初始姿态来实时估计目标姿态。 我们通过模拟和实际测量全面评估了两个模块的性能。结果表明，我们的无标识物系统在配准方面实现了比工业解决方案更优的表现，在跟踪方面的表现也与之相当。这种双模块设计使我们的系统成为适用于手术过程中目标解剖结构移动或静止的一站式解决方案。
URL
https://arxiv.org/abs/2504.09498
