Abstract
Recent advances in generative modeling have enabled the generation of high-quality synthetic data that is applicable in a variety of domains, including face recognition. Here, state-of-the-art generative models typically rely on conditioning and fine-tuning of powerful pretrained diffusion models to facilitate the synthesis of realistic images of a desired identity. Yet, these models often do not consider the identity of subjects during training, leading to poor consistency between generated and intended identities. In contrast, methods that employ identity-based training objectives tend to overfit on various aspects of the identity, and in turn, lower the diversity of images that can be generated. To address these issues, we present in this paper a novel generative diffusion-based framework, called ID-Booth. ID-Booth consists of a denoising network responsible for data generation, a variational auto-encoder for mapping images to and from a lower-dimensional latent space and a text encoder that allows for prompt-based control over the generation procedure. The framework utilizes a novel triplet identity training objective and enables identity-consistent image generation while retaining the synthesis capabilities of pretrained diffusion models. Experiments with a state-of-the-art latent diffusion model and diverse prompts reveal that our method facilitates better intra-identity consistency and inter-identity separability than competing methods, while achieving higher image diversity. In turn, the produced data allows for effective augmentation of small-scale datasets and training of better-performing recognition models in a privacy-preserving manner. The source code for the ID-Booth framework is publicly available at this https URL.
Abstract (translated)
近期在生成模型领域的进展使得能够为包括人脸识别在内的多个领域生成高质量的合成数据。这里，最先进的生成模型通常依赖于对强大的预训练扩散模型进行条件设置和微调，以促进所需身份的真实图像合成。然而，这些模型在训练时往往不考虑主体的身份，导致生成的身份与预期意图之间的一致性较差。相比之下，采用基于身份的训练目标的方法往往会过度拟合身份的各种方面，进而降低可以生成的图像多样性。 为了解决这些问题，我们在本文中提出了一种新的基于生成扩散框架的方法，称为ID-Booth（身份展位）。ID-Booth包括一个负责数据生成的去噪网络、将图像映射到和从较低维度潜在空间进行映射的变分自编码器以及允许根据提示控制生成过程的文本编码器。该框架利用了一种新颖的身份三元组训练目标，能够在保持预训练扩散模型合成能力的同时实现身份一致性的图像生成。 使用最先进的潜在扩散模型和各种不同的提示进行的实验表明，我们的方法比竞争的方法实现了更好的同身份一致性、跨身份可分离性以及更高的图像多样性。进而，所产生的数据可以有效增强小规模数据集，并以保护隐私的方式训练出更优秀的识别模型。 ID-Booth框架的源代码可在以下网址公开获得：[此链接处应填写实际URL]
URL
https://arxiv.org/abs/2504.07392
