Abstract
We present TIPS-GAN, a new approach to improve the accuracy and stability in unsupervised adversarial 2D to 3D human pose estimation. In our work we demonstrate that the human kinematic skeleton should not be assumed as one spatially dependent structure. In fact, we believe when a full 2D pose is provided during training, there is an inherent bias learned where the 3D coordinate of a keypoint is spatially codependent on the 2D locations of all other keypoints. To investigate our theory we follow previous adversarial approaches but trained two generators on spatially independent parts of the kinematic skeleton, the torso and the legs. During our study we find that improving self-consistency is key to lowering the evaluation error and therefore introduce new consistency constraints within the standard adversarial cycle. We then produced a final TIPS model via knowledge distillation which can predict the 3D coordinates for the entire 2D pose with improved results. Furthermore we help address the question left unanswered in prior adversarial learning papers of how long to train for a truly unsupervised scenario. We show that two independent generators training adversarially can hold a minimum error against a discriminator for a longer period of time than that of a solo generator which will diverge due to the adversarial network becoming unstable. TIPS decreases the average error by 18\% when compared to that of a baseline solo generator. TIPS improves upon other unsupervised approaches while also performing strongly against supervised and weakly-supervised approaches during evaluation on both the Human3.6M and MPI-INF-3DHP dataset.
Abstract (translated)
URL
https://arxiv.org/abs/2205.05980
