Abstract
Automated feedback systems have the potential to provide objective skill assessment for training and evaluation in robot-assisted surgery. In this study, we examine methods to achieve real-time prediction of surgical skill level in real-time based on Objective Structured Assessment of Technical Skills (OSATS) scores. Using data acquired from the da Vinci Surgical System, we carry out three main analyses, focusing on model design, their real-time performance, and their skill-level-based cross-validation training. For the model design, we evaluate the effectiveness of multimodal deep learning models for predicting surgical skill levels using synchronized kinematic and vision data. Our models include separate unimodal baselines and fusion architectures that integrate features from both modalities and are evaluated using mean Spearman's correlation coefficients, demonstrating that the fusion model consistently outperforms unimodal models for real-time predictions. For the real-time performance, we observe the prediction's trend over time and highlight correlation with the surgeon's gestures. For the skill-level-based cross-validation, we separately trained models on surgeons with different skill levels, which showed that high-skill demonstrations allow for better performance than those trained on low-skilled ones and generalize well to similarly skilled participants. Our findings show that multimodal learning allows more stable fine-grained evaluation of surgical performance and highlights the value of expert-level training data for model generalization.
Abstract (translated)
自动化反馈系统有潜力为机器人辅助手术中的培训和评估提供客观的技能评定。在这项研究中，我们探讨了基于客观结构化技术评分（OSATS）得分实现实时预测外科医生技能水平的方法。利用达芬奇外科系统的数据，我们进行了三项主要分析：模型设计、其实时性能以及基于技能水平的交叉验证训练。 在模型设计方面，我们评估了多模态深度学习模型的有效性，这些模型使用同步运动学和视觉数据来预测手术技能水平。我们的模型包括单一模式基准线及融合架构，后者整合了两种模式的特征，并通过平均Spearman相关系数进行评价。结果显示，融合模型在实时预测中始终优于单模式模型。 对于实时性能分析，我们观察了随时间变化的预测趋势，并强调与外科医生手势的相关性。在基于技能水平的交叉验证中，我们在不同技能水平的外科医生上分别训练了模型，结果表明，由高技能演示培训的模型比那些基于低技能数据训练的模型表现更好且能很好地泛化至类似技能水平的参与者。 我们的研究发现表明，多模态学习能够进行更为稳定细致的手术性能评估，并强调专家级训练数据对模型泛化的价值。
URL
https://arxiv.org/abs/2601.12012
