Abstract
Despite their potential, markerless hand tracking technologies are not yet applied in practice to the diagnosis or monitoring of the activity in inflammatory musculoskeletal diseases. One reason is that the focus of most methods lies in the reconstruction of coarse, plausible poses for gesture recognition or AR/VR applications, whereas in the clinical context, accurate, interpretable, and reliable results are required. Therefore, we propose ShaRPy, the first RGB-D Shape Reconstruction and hand Pose tracking system, which provides uncertainty estimates of the computed pose to guide clinical decision-making. Our method requires only a light-weight setup with a single consumer-level RGB-D camera yet it is able to distinguish similar poses with only small joint angle deviations. This is achieved by combining a data-driven dense correspondence predictor with traditional energy minimization, optimizing for both, pose and hand shape parameters. We evaluate ShaRPy on a keypoint detection benchmark and show qualitative results on recordings of a patient.
Abstract (translated)
尽管它们有潜力,但无标记手跟踪技术尚未在实践中应用于诊断或监测抗炎神经肌肉疾病的活动。原因之一是大多数方法的关注点在于对手势识别或增强现实应用中的粗略POS重建,而在实践中,需要准确、可解释和可靠的结果。因此,我们提出了SARPy,它是RGB-D形状重建和手POS跟踪系统的先驱,可以提供计算POS的不确定性估计,以指导临床决策。我们的方法只需要一个轻便的框架和一个消费级RGB-D相机,但它能够在仅有较小关节角度差异的情况下区分相似的POS。这是通过将数据驱动的密集对应预测与传统的能量最小化优化相结合实现的。我们评估了SARPy在一个关键点检测基准上的性能,并展示了患者记录中的质量结果。
URL
https://arxiv.org/abs/2303.10042
