Abstract
Identity-preserving face synthesis aims to generate synthetic face images of virtual subjects that can substitute real-world data for training face recognition models. While prior arts strive to create images with consistent identities and diverse styles, they face a trade-off between them. Identifying their limitation of treating style variation as subject-agnostic and observing that real-world persons actually have distinct, subject-specific styles, this paper introduces MorphFace, a diffusion-based face generator. The generator learns fine-grained facial styles, e.g., shape, pose and expression, from the renderings of a 3D morphable model (3DMM). It also learns identities from an off-the-shelf recognition model. To create virtual faces, the generator is conditioned on novel identities of unlabeled synthetic faces, and novel styles that are statistically sampled from a real-world prior distribution. The sampling especially accounts for both intra-subject variation and subject distinctiveness. A context blending strategy is employed to enhance the generator's responsiveness to identity and style conditions. Extensive experiments show that MorphFace outperforms the best prior arts in face recognition efficacy.
Abstract (translated)
身份保持人脸合成的目标是生成虚拟人物的合成面部图像，这些图像可以替代真实世界的数据用于训练人脸识别模型。虽然先前的研究致力于创建具有恒定身份和多样化风格的图像，但它们在两者之间面临着权衡。本文识别了之前研究将风格变化视为与主体无关的问题，并注意到现实生活中的人实际上拥有独特的、特定于个体的风格。为此，本文引入了MorphFace，这是一种基于扩散模型的脸部生成器。 该生成器从3D可变形态模型（3DMM）的渲染中学习到精细的面部样式，例如形状、姿态和表情。它还通过现有的人脸识别模型来学习身份信息。为了创建虚拟面孔，该生成器会在未标记的合成人脸的新身份上进行条件设定，并且新风格是从现实世界的先验分布中统计抽样得到的。尤其是这种抽样方法考虑到了个体内部的变化以及不同主体之间的差异性。 采用了一种上下文融合策略来增强生成器对身份和风格条件的响应能力。广泛的实验表明，MorphFace在人脸识别效果方面超越了现有的最佳技术。
URL
https://arxiv.org/abs/2504.00430
