Abstract
Recent synthetic 3D human datasets for the face, body, and hands have pushed the limits on photorealism. Face recognition and body pose estimation have achieved state-of-the-art performance using synthetic training data alone, but for the hand, there is still a large synthetic-to-real gap. This paper presents the first systematic study of the synthetic-to-real gap of 3D hand pose estimation. We analyze the gap and identify key components such as the forearm, image frequency statistics, hand pose, and object occlusions. To facilitate our analysis, we propose a data synthesis pipeline to synthesize high-quality data. We demonstrate that synthetic hand data can achieve the same level of accuracy as real data when integrating our identified components, paving the path to use synthetic data alone for hand pose estimation. Code and data are available at: this https URL.
Abstract (translated)
最近的合成3D人体数据集，包括面部、身体和手部，在逼真度方面取得了突破。使用仅基于合成训练数据的方法已经实现了面部识别和姿态估计领域的最先进性能，但对于手部而言，真实世界与合成数据之间的差距仍然很大。本文首次系统地研究了3D手部姿态估计中的真实世界与合成数据的差距，并分析了这一差距背后的关键因素，如前臂、图像频谱统计信息、手的姿态以及物体遮挡。 为了支持我们的分析，我们提出了一种数据合成分管道，用于生成高质量的数据。我们证明，在整合上述关键组件后，合成的手部数据可以达到与真实世界数据相同的准确度水平，从而为仅使用合成数据进行手部姿态估计铺平了道路。相关代码和数据可在以下网址获得：this https URL.
URL
https://arxiv.org/abs/2503.19307
