Abstract
Face recognition under extreme head poses is a challenging task. Ideally, a face recognition system should perform well across different head poses, which is known as pose-invariant face recognition. To achieve pose invariance, current approaches rely on sophisticated methods, such as face frontalization and various facial feature extraction model architectures. However, these methods are somewhat impractical in real-life settings and are typically evaluated on small scientific datasets, such as Multi-PIE. In this work, we propose the inverse method of face frontalization, called face defrontalization, to augment the training dataset of facial feature extraction model. The method does not introduce any time overhead during the inference step. The method is composed of: 1) training an adapted face defrontalization FFWM model on a frontal-profile pairs dataset, which has been preprocessed using our proposed face alignment method; 2) training a ResNet-50 facial feature extraction model based on ArcFace loss on a raw and randomly defrontalized large-scale dataset, where defrontalization was performed with our previously trained face defrontalization model. Our method was compared with the existing approaches on four open-access datasets: LFW, AgeDB, CFP, and Multi-PIE. Defrontalization shows improved results compared to models without defrontalization, while the proposed adjustments show clear superiority over the state-of-the-art face frontalization FFWM method on three larger open-access datasets, but not on the small Multi-PIE dataset for extreme poses (75 and 90 degrees). The results suggest that at least some of the current methods may be overfitted to small datasets.
Abstract (translated)
极端姿态下的面部识别是一个具有挑战性的任务。理想情况下，一个面部识别系统应该能够在不同的头部姿势下表现良好，这被称为无姿态变化的面部识别。为了实现这一目标，目前的方法依赖于复杂的技术，如面部正面化和各种面部特征提取模型架构。然而，在实际应用中这些方法往往不够实用，并且通常在小规模科学数据集（如Multi-PIE）上进行评估。 我们提出了一种逆向技术——面部非正面化(defrontalization)，用来增强面部特征提取模型的训练数据集。该方法不会增加推理步骤的时间开销。具体来说，我们的方法包括以下两个部分： 1. 在预先处理过的脸部对齐的数据集中（使用我们提出的面部对齐方法），通过训练一个自适应面部非正面化(FFWM)模型。 2. 基于ArcFace损失函数，在原始和随机非正面化的大型数据集上训练ResNet-50面部特征提取模型，其中非正面化过程由我们之前训练的面部非正面化模型执行。 我们将该方法与现有技术进行了比较，并在四个开源数据集中进行了测试：LFW、AgeDB、CFP以及Multi-PIE。实验结果显示，在三个更大的开放访问数据集上，我们的方法相对于没有进行非正面化的模型和最新的面部正面化FFWM方法表现出明显的优越性；但在极端姿势（75度和90度）下的小型Multi-PIE数据集中并没有显示出优势。 这些结果表明，至少部分当前的方法可能在小规模数据集上存在过拟合的问题。
URL
https://arxiv.org/abs/2506.04496
