Abstract
Purpose: Accurate detection and 6D pose estimation of surgical instruments are crucial for many computer-assisted interventions. However, supervised methods lack flexibility for new or unseen tools and require extensive annotated data. This work introduces a training-free pipeline for accurate multi-view 6D pose estimation of unseen surgical instruments, which only requires a textured CAD model as prior knowledge. Methods: Our pipeline consists of two main stages. First, for detection, we generate object mask proposals in each view and score their similarity to rendered templates using a pre-trained feature extractor. Detections are matched across views, triangulated into 3D instance candidates, and filtered using multi-view geometric consistency. Second, for pose estimation, a set of pose hypotheses is iteratively refined and scored using feature-metric scores with cross-view attention. The best hypothesis undergoes a final refinement using a novel multi-view, occlusion-aware contour registration, which minimizes reprojection errors of unoccluded contour points. Results: The proposed method was rigorously evaluated on real-world surgical data from the MVPSP dataset. The method achieves millimeter-accurate pose estimates that are on par with supervised methods under controlled conditions, while maintaining full generalization to unseen instruments. These results demonstrate the feasibility of training-free, marker-less detection and tracking in surgical scenes, and highlight the unique challenges in surgical environments. Conclusion: We present a novel and flexible pipeline that effectively combines state-of-the-art foundational models, multi-view geometry, and contour-based refinement for high-accuracy 6D pose estimation of surgical instruments without task-specific training. This approach enables robust instrument tracking and scene understanding in dynamic clinical environments.
Abstract (translated)
目的：精确检测和6D姿态估计对于许多计算机辅助介入手术至关重要。然而，监督方法缺乏对新工具或未见工具的灵活性，且需要大量标注数据。本研究提出了一种针对未见手术器械的免训练多视图6D姿态估计流程，仅需带纹理的CAD模型作为先验知识。 方法：我们的流程包含两个主要阶段。首先，在检测阶段，我们在每个视图中生成目标掩码提案，并使用预训练的特征提取器评估其与渲染模板的相似性。检测结果跨视图匹配，三角化为3D实例候选，并通过多视图几何一致性进行过滤。其次，在姿态估计阶段，一组姿态假设通过跨视图注意力的特征度量分数进行迭代优化和评分。最佳假设采用一种新颖的多视图、感知遮挡的轮廓配准方法进行最终优化，该方法最小化未遮挡轮廓点的重投影误差。 结果：所提方法在MVPSP数据集的真实手术数据上进行了严格评估。该方法实现了毫米级精度的姿态估计，在受控条件下与监督方法相当，同时保持对未见器械的完全泛化能力。这些结果证明了手术场景中免训练、无标记检测与跟踪的可行性，并凸显了手术环境中的独特挑战。 结论：我们提出了一种新颖灵活的流程，有效结合了最先进的基座模型、多视图几何和基于轮廓的优化，无需任务特定训练即可实现手术器械的高精度6D姿态估计。该方法支持动态临床环境中的稳健器械跟踪与场景理解。
URL
https://arxiv.org/abs/2603.25228
