Abstract
Face recognition (FR) stands as one of the most crucial applications in computer vision. The accuracy of FR models has significantly improved in recent years due to the availability of large-scale human face datasets. However, directly using these datasets can inevitably lead to privacy and legal problems. Generating synthetic data to train FR models is a feasible solution to circumvent these issues. While existing synthetic-based face recognition methods have made significant progress in generating identity-preserving images, they are severely plagued by context overfitting, resulting in a lack of intra-class diversity of generated images and poor face recognition performance. In this paper, we propose a framework to Unleash Inherent capability of the model to enhance intra-class diversity for synthetic face recognition, shortened as UIFace. Our framework first trains a diffusion model that can perform sampling conditioned on either identity contexts or a learnable empty context. The former generates identity-preserving images but lacks variations, while the latter exploits the model's intrinsic ability to synthesize intra-class-diversified images but with random identities. Then we adopt a novel two-stage sampling strategy during inference to fully leverage the strengths of both types of contexts, resulting in images that are diverse as well as identitypreserving. Moreover, an attention injection module is introduced to further augment the intra-class variations by utilizing attention maps from the empty context to guide the sampling process in ID-conditioned generation. Experiments show that our method significantly surpasses previous approaches with even less training data and half the size of synthetic dataset. The proposed UIFace even achieves comparable performance with FR models trained on real datasets when we further increase the number of synthetic identities.
Abstract (translated)
人脸识别（FR）是计算机视觉领域中最重要的应用之一。近年来，由于大规模人脸数据集的可用性，人脸识别模型的准确性得到了显著提高。然而，直接使用这些数据集不可避免地会导致隐私和法律问题。生成合成数据以训练人脸识别模型是一种可行的方法来避免这些问题。虽然现有的基于合成的人脸识别方法在生成身份保持图像方面已经取得了重大进展，但它们仍然严重受到上下文过拟合的影响，导致生成的图像缺乏类内多样性，并且面部识别性能较差。 为此，在本文中我们提出了一种框架，旨在通过增强合成人脸数据的类内多样性来提升模型的内在能力，简称UIFace。我们的框架首先训练一个扩散模型，该模型可以在身份上下文或可学习的空上下文中进行采样。前者可以生成身份保持图像但缺乏变化性，后者则能够利用模型固有的能力以随机的身份生成类内多样化的图像。然后我们在推理阶段采用一种新颖的两阶段采样策略，充分利用这两种上下文的优点，从而产生多样化且身份保持的图像。 此外，我们引入了一个注意力注入模块，通过使用空上下文的注意力图来指导身份条件下的生成过程，进一步增强类内变化性。实验结果显示，即使在训练数据更少和合成数据集规模减半的情况下，我们的方法仍然显著优于先前的方法。当增加更多的合成身份数量时，UIFace甚至能够实现与基于真实数据集的人脸识别模型相当的性能。
URL
https://arxiv.org/abs/2502.19803
