Abstract
Automated Face Recognition Systems (FRSs), developed using deep learning models, are deployed worldwide for identity verification and facial attribute analysis. The performance of these models is determined by a complex interdependence among the model architecture, optimization/loss function and datasets. Although FRSs have surpassed human-level accuracy, they continue to be disparate against certain demographics. Due to the ubiquity of applications, it is extremely important to understand the impact of the three components -- model architecture, loss function and face image dataset on the accuracy-disparity trade-off to design better, unbiased platforms. In this work, we perform an in-depth analysis of three FRSs for the task of gender prediction, with various architectural modifications resulting in ten deep-learning models coupled with four loss functions and benchmark them on seven face datasets across 266 evaluation configurations. Our results show that all three components have an individual as well as a combined impact on both accuracy and disparity. We identify that datasets have an inherent property that causes them to perform similarly across models, independent of the choice of loss functions. Moreover, the choice of dataset determines the model's perceived bias -- the same model reports bias in opposite directions for three gender-balanced datasets of ``in-the-wild'' face images of popular individuals. Studying the facial embeddings shows that the models are unable to generalize a uniform definition of what constitutes a ``female face'' as opposed to a ``male face'', due to dataset diversity. We provide recommendations to model developers on using our study as a blueprint for model development and subsequent deployment.
Abstract (translated)
使用深度学习模型开发的自动化面部识别系统（FRS）在全球范围内用于身份验证和面部属性分析。这些模型的表现由其架构、优化/损失函数以及数据集之间的复杂相互作用决定。尽管FRS已经超过了人类级别的准确度，但在某些人口群体中依然存在显著差异。鉴于应用的广泛性，理解这三个组成部分——即模型架构、损失函数及面部图像数据集如何影响准确性与偏差权衡至关重要，以便设计更公正、无偏见的平台。 在这项工作中，我们对三个用于性别预测任务的FRS进行了深入分析，在此基础上通过各种架构修改生成了十个深度学习模型，并结合四种不同的损失函数在七个人脸数据集上进行测试，涵盖266种评估配置。我们的结果显示，这三个组成部分无论是单独还是组合起来都对准确性和偏差有着显著影响。 我们发现，某些数据集具有固有的特性，导致它们在不同模型之间表现出一致的行为，这与所选择的损失函数无关。此外，数据集的选择决定了模型被感知到的偏见——同样的模型在三个性别均衡的“真实世界”名人面部图像数据集中会显示出相反方向上的偏差。 研究面部嵌入显示，由于数据集的多样性，这些模型无法形成统一定义什么是“女性脸”与“男性脸”，从而导致了泛化能力的问题。我们为模型开发者提供了基于本研究结果进行模型开发和后续部署的建议。
URL
https://arxiv.org/abs/2503.14138
