Abstract
Head pose estimation (HPE) plays a critical role in various computer vision applications such as human-computer interaction and facial recognition. In this paper, we propose a novel deep learning approach for head pose estimation with limited training data via non-linear manifold learning called NLML-HPE. This method is based on the combination of tensor decomposition (i.e., Tucker decomposition) and feed forward neural networks. Unlike traditional classification-based approaches, our method formulates head pose estimation as a regression problem, mapping input landmarks into a continuous representation of pose angles. To this end, our method uses tensor decomposition to split each Euler angle (yaw, pitch, roll) to separate subspaces and models each dimension of the underlying manifold as a cosine curve. We address two key challenges: 1. Almost all HPE datasets suffer from incorrect and inaccurate pose annotations. Hence, we generated a precise and consistent 2D head pose dataset for our training set by rotating 3D head models for a fixed set of poses and rendering the corresponding 2D images. 2. We achieved real-time performance with limited training data as our method accurately captures the nature of rotation of an object from facial landmarks. Once the underlying manifold for rotation around each axis is learned, the model is very fast in predicting unseen data. Our training and testing code is available online along with our trained models: https: //github.com/MahdiGhafoorian/NLML_HPE.
Abstract (translated)
头部姿态估计（HPE）在人机交互和面部识别等众多计算机视觉应用中扮演着关键角色。本文提出了一种新颖的深度学习方法，通过非线性流形学习（NLML-HPE），利用有限训练数据进行头部姿态估计。该方法基于张量分解（即Tucker分解）与前馈神经网络相结合。 不同于传统的分类方法，我们的方法将头部姿态估计视为回归问题，并将输入标记映射到连续的姿态角度表示中。为此，我们使用张量分解将每个欧拉角（偏航角、俯仰角和滚转角）拆分为独立的子空间，并将潜在流形的每一维度建模为余弦曲线。 本研究解决了两个主要挑战： 1. 几乎所有头部姿态数据集都存在错误或不准确的姿态标注。为此，我们通过旋转固定姿势集合中的3D头模型并渲染相应的2D图像来生成精确且一致的二维头部姿态训练数据集。 2. 我们在有限的训练数据情况下实现了实时性能，因为我们的方法能够从面部标记中精确捕捉物体旋转的本质特性。一旦学习了每个轴上的旋转潜在流形，该模型就能快速预测未见过的数据。 我们提供的训练和测试代码以及预训练模型可以在线获取：https://github.com/MahdiGhafoorian/NLML_HPE
URL
https://arxiv.org/abs/2507.18429
