Abstract
This work proposes a novel learning framework for visual hand dynamics analysis that takes into account the physiological aspects of hand motion. The existing models, which are simplified joint-actuated systems, often produce unnatural motions. To address this, we integrate a musculoskeletal system with a learnable parametric hand model, MANO, to create a new model, MS-MANO. This model emulates the dynamics of muscles and tendons to drive the skeletal system, imposing physiologically realistic constraints on the resulting torque trajectories. We further propose a simulation-in-the-loop pose refinement framework, BioPR, that refines the initial estimated pose through a multi-layer perceptron (MLP) network. Our evaluation of the accuracy of MS-MANO and the efficacy of the BioPR is conducted in two separate parts. The accuracy of MS-MANO is compared with MyoSuite, while the efficacy of BioPR is benchmarked against two large-scale public datasets and two recent state-of-the-art methods. The results demonstrate that our approach consistently improves the baseline methods both quantitatively and qualitatively.
Abstract (translated)
本文提出了一种新的视觉手动态分析学习框架，考虑了手运动的生理学方面。现有的模型，这些模型是简化的关节驱动系统，通常会产生不自然运动。为了解决这个问题，我们将一个肌肉骨骼系统与可学习参数化手模型MANO集成，以创建一个新的模型MS-MANO。这个模型模拟了肌肉和肌腱的运动，驱动骨骼系统，对结果的扭矩轨迹施加生理学上的约束。我们还提出了一个模拟-在-循环姿势优化框架BioPR，该框架通过多层感知器（MLP）网络对初始估计姿势进行优化。我们对MS-MANO和BioPR的准确性和有效性进行了两个部分的评估。MS-MANO的准确性与MyoSuite进行了比较，BioPR的有效性则与两个大型公开数据集和两个最近的方法进行了比较。结果表明，我们的方法在数量和质量上都能显著提高基准方法。
URL
https://arxiv.org/abs/2404.10227
