Face_Recognition
-
GANDiff FR: Hybrid GAN Diffusion Synthesis for Causal Bias Attribution in Face Recognition
2025-08-15 09:05:57Md Asgor Hossain Reaj, Rajan Das Gupta, Md Yeasin Rahat, Nafiz Fahad, Md Jawadul Hasan, Tze Hui LiewAbstract
We introduce GANDiff FR, the first synthetic framework that precisely controls demographic and environmental factors to measure, explain, and reduce bias with reproducible rigor. GANDiff FR unifies StyleGAN3-based identity-preserving generation with diffusion-based attribute control, enabling fine-grained manipulation of pose around 30 degrees, illumination (four directions), and expression (five levels) under ceteris paribus conditions. We synthesize 10,000 demographically balanced faces across five cohorts validated for realism via automated detection (98.2%) and human review (89%) to isolate and quantify bias drivers. Benchmarking ArcFace, CosFace, and AdaFace under matched operating points shows AdaFace reduces inter-group TPR disparity by 60% (2.5% vs. 6.3%), with illumination accounting for 42% of residual bias. Cross-dataset evaluation on RFW, BUPT, and CASIA WebFace confirms strong synthetic-to-real transfer (r 0.85). Despite around 20% computational overhead relative to pure GANs, GANDiff FR yields three times more attribute-conditioned variants, establishing a reproducible, regulation-aligned (EU AI Act) standard for fairness auditing. Code and data are released to support transparent, scalable bias evaluation.
Abstract (translated)
我们介绍了GANDiff FR，这是第一个能够精确控制人口统计学和环境因素的合成框架，用于测量、解释和减少偏差，并且具有可重复验证的严谨性。GANDiff FR结合了基于StyleGAN3的身份保持生成与基于扩散的方法属性控制，使得在其他条件不变的情况下，可以精细调整姿态（约30度范围内）、照明（四个方向）以及表情（五个级别）。我们合成了10,000张跨五类群体的人口统计学平衡面部图像，并通过自动检测（98.2%）和人工审核（89%）验证了其真实感，以便隔离并量化偏差驱动因素。在匹配的操作点下对ArcFace、CosFace和AdaFace进行基准测试显示，AdaFace将组间TPR差异减少了60%（2.5% vs 6.3%），其中照明占剩余偏见的42%。跨数据集评估证明了RFW、BUPT和CASIA WebFace中强大的合成到真实图像转移能力（相关系数r为0.85）。尽管相对于纯GANs，GANDiff FR在计算上增加了约20%的开销，但它生成的属性条件变体是纯GAN的三倍，并建立了符合欧盟人工智能法案等监管标准的可重复且公平审计的标准。代码和数据已发布以支持透明、可扩展的偏差评估。
URL
https://arxiv.org/abs/2508.11334
PDF
-
Hybrid Generative Fusion for Efficient and Privacy-Preserving Face Recognition Dataset Generation
2025-08-14 14:14:18Feiran Li, Qianqian Xu, Shilong Bao, Boyu Han, Zhiyong Yang, Qingming HuangAbstract
In this paper, we present our approach to the DataCV ICCV Challenge, which centers on building a high-quality face dataset to train a face recognition model. The constructed dataset must not contain identities overlapping with any existing public face datasets. To handle this challenge, we begin with a thorough cleaning of the baseline HSFace dataset, identifying and removing mislabeled or inconsistent identities through a Mixture-of-Experts (MoE) strategy combining face embedding clustering and GPT-4o-assisted verification. We retain the largest consistent identity cluster and apply data augmentation up to a fixed number of images per identity. To further diversify the dataset, we generate synthetic identities using Stable Diffusion with prompt engineering. As diffusion models are computationally intensive, we generate only one reference image per identity and efficiently expand it using Vec2Face, which rapidly produces 49 identity-consistent variants. This hybrid approach fuses GAN-based and diffusion-based samples, enabling efficient construction of a diverse and high-quality dataset. To address the high visual similarity among synthetic identities, we adopt a curriculum learning strategy by placing them early in the training schedule, allowing the model to progress from easier to harder samples. Our final dataset contains 50 images per identity, and all newly generated identities are checked with mainstream face datasets to ensure no identity leakage. Our method achieves \textbf{1st place} in the competition, and experimental results show that our dataset improves model performance across 10K, 20K, and 100K identity scales. Code is available at this https URL.
Abstract (translated)
在这篇论文中，我们介绍了我们在DataCV ICCV挑战赛中的方法，该挑战旨在建立一个高质量的面部数据集以训练人脸识别模型。构建的数据集中不能包含任何现有公共面部数据集中已有的身份信息。为了解决这个挑战，我们首先对基础HSFace数据集进行了彻底清理，通过混合专家（MoE）策略结合面部嵌入聚类和GPT-4辅助验证来识别并移除错误标注或不一致的身份信息。我们保留了最大的一致性身份簇，并对每个身份应用最多固定数量的图像进行数据增强。 为了进一步丰富数据集的多样性，我们使用Stable Diffusion与提示工程生成合成身份。由于扩散模型计算密集型的特点，我们为每个身份仅生成一个参考图像并利用Vec2Face高效扩展它，快速生成49个一致性变体。这种混合方法将基于GAN和扩散模型的数据样本结合在一起，实现了一个多样性和高质量数据集的高效构建。 为了处理合成身份之间的高视觉相似性问题，我们在训练时间表中早期放置它们，并采用课程学习策略，使模型能够从简单到复杂的样例进行学习。最终的数据集包含每个身份50张图像，所有新生成的身份都经过了与主流面部数据集的检查以确保没有身份信息泄露。 我们的方法在比赛中获得了**第一名**的成绩，实验结果表明，我们构建的数据集在10K、20K和100K个身份规模上均提高了模型性能。代码可以在[此链接](https://this https URL)获取（请将"this https URL"替换为实际的URL地址）。
URL
https://arxiv.org/abs/2508.10672
PDF
-
Empowering Morphing Attack Detection using Interpretable Image-Text Foundation Model
2025-08-13 18:06:29Sushrut Patwardhan, Raghavendra Ramachandra, Sushma VenkateshAbstract
Morphing attack detection has become an essential component of face recognition systems for ensuring a reliable verification scenario. In this paper, we present a multimodal learning approach that can provide a textual description of morphing attack detection. We first show that zero-shot evaluation of the proposed framework using Contrastive Language-Image Pretraining (CLIP) can yield not only generalizable morphing attack detection, but also predict the most relevant text snippet. We present an extensive analysis of ten different textual prompts that include both short and long textual prompts. These prompts are engineered by considering the human understandable textual snippet. Extensive experiments were performed on a face morphing dataset that was developed using a publicly available face biometric dataset. We present an evaluation of SOTA pre-trained neural networks together with the proposed framework in the zero-shot evaluation of five different morphing generation techniques that are captured in three different mediums.
Abstract (translated)
面部变形攻击检测已成为确保面部识别系统可靠验证场景的重要组成部分。本文提出了一种多模态学习方法，可以为变形攻击检测提供文本描述。首先展示使用对比语言-图像预训练（CLIP）对所提框架进行零样本评估不仅可以实现泛化的变形攻击检测，还能预测最相关的文本片段。我们详细分析了十种不同的文本提示，包括短文本和长文本提示，这些提示是根据人类可理解的文本片段设计的。在使用公开可用的人脸生物识别数据集开发的脸部变形数据集上进行了大量实验。我们在零样本评估中对五种不同生成技术（以三种不同媒介捕获）的最先进预训练神经网络以及所提出的框架进行了一项评价。 该研究揭示了通过对比语言-图像预训练模型能够有效实现面部变形攻击检测，并且可以输出相应的文本描述，进一步提高系统的解释性和实用性。
URL
https://arxiv.org/abs/2508.10110
PDF
-
NegFaceDiff: The Power of Negative Context in Identity-Conditioned Diffusion for Synthetic Face Generation
2025-08-13 09:45:09Eduarda Caldeira, Naser Damer, Fadi BoutrosAbstract
The use of synthetic data as an alternative to authentic datasets in face recognition (FR) development has gained significant attention, addressing privacy, ethical, and practical concerns associated with collecting and using authentic data. Recent state-of-the-art approaches have proposed identity-conditioned diffusion models to generate identity-consistent face images, facilitating their use in training FR models. However, these methods often lack explicit sampling mechanisms to enforce inter-class separability, leading to identity overlap in the generated data and, consequently, suboptimal FR performance. In this work, we introduce NegFaceDiff, a novel sampling method that incorporates negative conditions into the identity-conditioned diffusion process. NegFaceDiff enhances identity separation by leveraging negative conditions that explicitly guide the model away from unwanted features while preserving intra-class consistency. Extensive experiments demonstrate that NegFaceDiff significantly improves the identity consistency and separability of data generated by identity-conditioned diffusion models. Specifically, identity separability, measured by the Fisher Discriminant Ratio (FDR), increases from 2.427 to 5.687. These improvements are reflected in FR systems trained on the NegFaceDiff dataset, which outperform models trained on data generated without negative conditions across multiple benchmarks.
Abstract (translated)
在人脸识别（FR）开发中，合成数据作为一种替代真实数据集的方法已经获得了广泛关注，这解决了收集和使用真实数据时出现的隐私、伦理以及实际操作方面的担忧。最近的研究前沿提出了身份条件扩散模型来生成一致的身份人脸图像，从而促进其用于训练人脸识别模型。然而，这些方法往往缺少明确的采样机制以强制执行跨类别的可分离性，导致生成的数据中存在身份重叠问题，并最终影响了人脸识别系统的性能。 在本研究中，我们引入了一种名为NegFaceDiff的新颖采样方法，该方法将负条件纳入到身份条件扩散过程中。通过使用负面条件明确引导模型远离不希望出现的特征同时保持同类内的一致性，NegFaceDiff增强了不同身份之间的分离度。广泛的实验表明，与没有负面条件的数据相比，使用NegFaceDiff生成的身份条件扩散数据在身份一致性和可分离性方面有了显著改善。具体来说，通过费舍尔判别比（FDR）测量的识别区分度从2.427提升到了5.687。 这些改进反映在基于NegFaceDiff数据集训练的人脸识别系统中，在多个基准测试上，此类模型的表现优于未使用负面条件生成的数据训练的模型。
URL
https://arxiv.org/abs/2508.09661
PDF
-
MADPromptS: Unlocking Zero-Shot Morphing Attack Detection with Multiple Prompt Aggregation
2025-08-12 13:47:27Eduarda Caldeira, Fadi Boutros, Naser DamerAbstract
Face Morphing Attack Detection (MAD) is a critical challenge in face recognition security, where attackers can fool systems by interpolating the identity information of two or more individuals into a single face image, resulting in samples that can be verified as belonging to multiple identities by face recognition systems. While multimodal foundation models (FMs) like CLIP offer strong zero-shot capabilities by jointly modeling images and text, most prior works on FMs for biometric recognition have relied on fine-tuning for specific downstream tasks, neglecting their potential for direct, generalizable deployment. This work explores a pure zero-shot approach to MAD by leveraging CLIP without any additional training or fine-tuning, focusing instead on the design and aggregation of multiple textual prompts per class. By aggregating the embeddings of diverse prompts, we better align the model's internal representations with the MAD task, capturing richer and more varied cues indicative of bona-fide or attack samples. Our results show that prompt aggregation substantially improves zero-shot detection performance, demonstrating the effectiveness of exploiting foundation models' built-in multimodal knowledge through efficient prompt engineering.
Abstract (translated)
面部变形攻击检测（MAD）是人脸识别安全中的一个关键挑战。在这种情况下，攻击者可以通过将两个或更多个人的身份信息插值到一张人脸图像中来欺骗系统，从而生成可以被验证为属于多个身份的人脸识别样本。虽然多模态基础模型（如CLIP）通过共同建模图像和文本提供了强大的零样本能力，但大多数先前关于生物特征识别中的FMs的工作依赖于针对特定下游任务的微调，而忽略了它们直接通用部署的潜力。这项工作探索了一种纯零样本方法来解决MAD问题，即在不进行任何额外训练或微调的情况下利用CLIP，而是专注于每个类别的多种文本提示的设计和聚合。通过聚合多样化的提示嵌入，我们可以更好地将模型内部表示与MAD任务对齐，捕捉更多更丰富的指示真实样本或攻击样本的线索。 我们的实验结果表明，提示聚合显著提高了零样本检测性能，证明了通过高效的提示工程来利用基础模型内置的多模态知识的有效性。
URL
https://arxiv.org/abs/2508.08939
PDF
-
Identity-Preserving Aging and De-Aging of Faces in the StyleGAN Latent Space
2025-08-12 10:02:42Luis S. Luevano, Pavel Korshunov, Sebastien MarcelAbstract
Face aging or de-aging with generative AI has gained significant attention for its applications in such fields like forensics, security, and media. However, most state of the art methods rely on conditional Generative Adversarial Networks (GANs), Diffusion-based models, or Visual Language Models (VLMs) to age or de-age faces based on predefined age categories and conditioning via loss functions, fine-tuning, or text prompts. The reliance on such conditioning leads to complex training requirements, increased data needs, and challenges in generating consistent results. Additionally, identity preservation is rarely taken into accountor evaluated on a single face recognition system without any control or guarantees on whether identity would be preserved in a generated aged/de-aged face. In this paper, we propose to synthesize aged and de-aged faces via editing latent space of StyleGAN2 using a simple support vector modeling of aging/de-aging direction and several feature selection approaches. By using two state-of-the-art face recognition systems, we empirically find the identity preserving subspace within the StyleGAN2 latent space, so that an apparent age of a given face can changed while preserving the identity. We then propose a simple yet practical formula for estimating the limits on aging/de-aging parameters that ensures identity preservation for a given input face. Using our method and estimated parameters we have generated a public dataset of synthetic faces at different ages that can be used for benchmarking cross-age face recognition, age assurance systems, or systems for detection of synthetic images. Our code and dataset are available at the project page this https URL
Abstract (translated)
使用生成人工智能进行面部老化或逆向老化技术，在法医、安全和媒体等领域得到了广泛的关注。然而，大多数最先进的方法依赖于条件生成对抗网络（GAN）、基于扩散的模型或视觉语言模型（VLM），通过预定义的年龄类别以及损失函数、微调或文本提示等方式来进行面部的老化或逆向老化处理。这种依赖条件设定的方法会导致复杂的训练要求，增加数据需求，并且在生成一致结果方面存在挑战。此外，在单一人脸识别系统中很少考虑或评估身份保持问题，即无法保证生成的老化或逆向老化后的脸部图像能够保留原有的身份信息。 在这篇论文中，我们提出了一种通过编辑StyleGAN2的潜在空间来合成老化和逆向老化的面部的方法。我们在潜在空间中使用了简单的支持向量模型来描述老化/逆向老化的方向，并采用了几种特征选择方法。通过利用两种最先进的面部识别系统，我们从实验上找到了能够保持身份一致性的子空间，使给定的面部图像能够在改变外观年龄的同时保留其身份信息。随后，我们提出了一种简单的计算公式，用于估算确保在处理过程中身份不会丢失的老化/逆向老化参数范围。使用我们的方法和估计出的参数，我们生成了一个公开的人脸合成数据集，包含了不同年龄段的数据样本，可用于跨年龄段人脸识别、年龄验证系统或检测合成图像系统的基准测试。 项目的代码和数据集可以在以下链接获取：[请在此处填写实际提供的项目页面URL]
URL
https://arxiv.org/abs/2508.08808
PDF
-
VOIDFace: A Privacy-Preserving Multi-Network Face Recognition With Enhanced Security
2025-08-11 13:15:36Ajnas Muhammed, Iurri Medvedev, Nuno Gon\c{c}alvesAbstract
Advancement of machine learning techniques, combined with the availability of large-scale datasets, has significantly improved the accuracy and efficiency of facial recognition. Modern facial recognition systems are trained using large face datasets collected from diverse individuals or public repositories. However, for training, these datasets are often replicated and stored in multiple workstations, resulting in data replication, which complicates database management and oversight. Currently, once a user submits their face for dataset preparation, they lose control over how their data is used, raising significant privacy and ethical concerns. This paper introduces VOIDFace, a novel framework for facial recognition systems that addresses two major issues. First, it eliminates the need of data replication and improves data control to securely store training face data by using visual secret sharing. Second, it proposes a patch-based multi-training network that uses this novel training data storage mechanism to develop a robust, privacy-preserving facial recognition system. By integrating these advancements, VOIDFace aims to improve the privacy, security, and efficiency of facial recognition training, while ensuring greater control over sensitive personal face data. VOIDFace also enables users to exercise their Right-To-Be-Forgotten property to control their personal data. Experimental evaluations on the VGGFace2 dataset show that VOIDFace provides Right-To-Be-Forgotten, improved data control, security, and privacy while maintaining competitive facial recognition performance. Code is available at: this https URL
Abstract (translated)
机器学习技术的进步，结合大规模数据集的可用性，显著提高了面部识别的准确性和效率。现代面部识别系统使用从各种个体或公共存储库收集的大规模人脸数据进行训练。然而，在训练过程中，这些数据集通常被复制并存储在多个工作站上，导致了数据冗余，并且使数据库管理变得复杂化。目前的情况是，一旦用户提交他们的面部信息用于数据集准备，他们就失去了对其数据使用方式的控制权，这引发了重要的隐私和伦理问题。 本文介绍了VOIDFace框架，这是一个面向面部识别系统的创新方案，旨在解决两大主要问题：首先，它消除了数据复制的需求，并通过采用视觉秘密共享技术来提高数据管理的安全性，以安全地存储训练用的人脸数据。其次，它提出了一种基于补丁的多训练网络，利用这种新颖的数据存储机制来开发一种既强大又保护隐私的面部识别系统。 通过整合这些改进措施，VOIDFace旨在提升面部识别系统的隐私、安全性和效率，并确保对敏感个人面部数据拥有更大的控制权。此外，VOIDFace还允许用户行使“被遗忘的权利”，即有权控制和管理自己的个人信息。 在VGGFace2数据集上的实验评估表明，VOIDFace不仅能提供符合“被遗忘的权利”的能力，还能改进数据控制、安全性和隐私保护，并且仍然保持竞争力的面部识别性能。 代码可在以下网址获取：[此链接](https://this-url.com)
URL
https://arxiv.org/abs/2508.07960
PDF
-
FaceAnonyMixer: Cancelable Faces via Identity Consistent Latent Space Mixing
2025-08-07 17:59:59Mohammed Talha Alam, Fahad Shamshad, Fakhri Karray, Karthik NandakumarAbstract
Advancements in face recognition (FR) technologies have amplified privacy concerns, necessitating methods that protect identity while maintaining recognition utility. Existing face anonymization methods typically focus on obscuring identity but fail to meet the requirements of biometric template protection, including revocability, unlinkability, and irreversibility. We propose FaceAnonyMixer, a cancelable face generation framework that leverages the latent space of a pre-trained generative model to synthesize privacy-preserving face images. The core idea of FaceAnonyMixer is to irreversibly mix the latent code of a real face image with a synthetic code derived from a revocable key. The mixed latent code is further refined through a carefully designed multi-objective loss to satisfy all cancelable biometric requirements. FaceAnonyMixer is capable of generating high-quality cancelable faces that can be directly matched using existing FR systems without requiring any modifications. Extensive experiments on benchmark datasets demonstrate that FaceAnonyMixer delivers superior recognition accuracy while providing significantly stronger privacy protection, achieving over an 11% gain on commercial API compared to recent cancelable biometric methods. Code is available at: this https URL.
Abstract (translated)
面部识别技术的进步引发了隐私问题，因此需要既能保护个人身份又能保持识别功能的方法。现有的面部匿名化方法通常专注于模糊身份信息，但未能满足生物特征模板保护的要求，包括可撤销性、不可链接性和不可逆性。我们提出了一种名为FaceAnonyMixer的框架，它利用预训练生成模型的潜在空间来合成具有隐私保护特性的面部图像。 FaceAnonyMixer的核心思想是将真实面部图像的潜在代码与从可撤销密钥中衍生出的合成代码进行不可逆混合。然后通过精心设计的多目标损失函数进一步优化混合后的潜在代码，以满足所有生物特征取消的要求。FaceAnonyMixer能够生成高质量且可以直接使用现有面部识别系统进行匹配的可撤销面部图像，并不需要对这些系统做任何修改。 在基准数据集上的广泛实验表明，FaceAnonyMixer能够在提供显著更强隐私保护的同时实现更优的人脸识别准确率，在商用API上相比最近的生物特征取消方法提升了超过11%。代码可以在以下链接获取：[此URL]（请将“this https URL”替换为实际提供的GitHub或其他版本控制平台上的链接）。
URL
https://arxiv.org/abs/2508.05636
PDF
-
From Detection to Correction: Backdoor-Resilient Face Recognition via Vision-Language Trigger Detection and Noise-Based Neutralization
2025-08-07 14:02:34Farah Wahida, M. A. P. Chamikara, Yashothara Shanmugarasa, Mohan Baruwal Chhetri, Thilina Ranbaduge, Ibrahim KhalilAbstract
Biometric systems, such as face recognition systems powered by deep neural networks (DNNs), rely on large and highly sensitive datasets. Backdoor attacks can subvert these systems by manipulating the training process. By inserting a small trigger, such as a sticker, make-up, or patterned mask, into a few training images, an adversary can later present the same trigger during authentication to be falsely recognized as another individual, thereby gaining unauthorized access. Existing defense mechanisms against backdoor attacks still face challenges in precisely identifying and mitigating poisoned images without compromising data utility, which undermines the overall reliability of the system. We propose a novel and generalizable approach, TrueBiometric: Trustworthy Biometrics, which accurately detects poisoned images using a majority voting mechanism leveraging multiple state-of-the-art large vision language models. Once identified, poisoned samples are corrected using targeted and calibrated corrective noise. Our extensive empirical results demonstrate that TrueBiometric detects and corrects poisoned images with 100\% accuracy without compromising accuracy on clean images. Compared to existing state-of-the-art approaches, TrueBiometric offers a more practical, accurate, and effective solution for mitigating backdoor attacks in face recognition systems.
Abstract (translated)
基于生物特征的系统（如由深度神经网络(DNN)支持的人脸识别系统）依赖于大量且高度敏感的数据集。后门攻击可以通过篡改训练过程来破坏这些系统，通过在少数几张训练图像中插入一个小触发器（例如贴纸、化妆或图案面具），对手可以在后续的身份验证过程中展示相同的触发器，从而被错误地识别为另一个个体，并获得未经授权的访问权限。 现有的防御机制仍然面临挑战，在准确识别和缓解受到污染的图片而不牺牲数据效用的同时保持系统的整体可靠性。我们提出了一种新颖且具有广泛适用性的方法——TrueBiometric：可信赖生物特征技术（Trustworthy Biometrics），它利用多个先进的大型视觉语言模型通过多数投票机制精确地检测受到污染的图像。一旦识别出受污染的样本，会使用定向和校准过的修正噪声进行纠正。 我们的大量实证研究表明，TrueBiometric能够以100%的准确率检测并修复被篡改的图片，同时不损害未受损图片的准确性。与现有的最先进的方法相比，TrueBiometric为减轻人脸识别系统中的后门攻击提供了一种更实用、精确和有效的解决方案。
URL
https://arxiv.org/abs/2508.05409
PDF
-
Protego: User-Centric Pose-Invariant Privacy Protection Against Face Recognition-Induced Digital Footprint Exposure
2025-08-04 04:03:01Ziling Wang, Shuya Yang, Jialin Lu, Ka-Ho ChowAbstract
Face recognition (FR) technologies are increasingly used to power large-scale image retrieval systems, raising serious privacy concerns. Services like Clearview AI and PimEyes allow anyone to upload a facial photo and retrieve a large amount of online content associated with that person. This not only enables identity inference but also exposes their digital footprint, such as social media activity, private photos, and news reports, often without their consent. In response to this emerging threat, we propose Protego, a user-centric privacy protection method that safeguards facial images from such retrieval-based privacy intrusions. Protego encapsulates a user's 3D facial signatures into a pose-invariant 2D representation, which is dynamically deformed into a natural-looking 3D mask tailored to the pose and expression of any facial image of the user, and applied prior to online sharing. Motivated by a critical limitation of existing methods, Protego amplifies the sensitivity of FR models so that protected images cannot be matched even among themselves. Experiments show that Protego significantly reduces retrieval accuracy across a wide range of black-box FR models and performs at least 2x better than existing methods. It also offers unprecedented visual coherence, particularly in video settings where consistency and natural appearance are essential. Overall, Protego contributes to the fight against the misuse of FR for mass surveillance and unsolicited identity tracing.
Abstract (translated)
面部识别（FR）技术日益被用于大型图像检索系统，引发了严重的隐私问题。像Clearview AI和PimEyes这样的服务允许任何人上传一张面部照片，并检索与此人相关的大量在线内容。这不仅使身份推断成为可能，还暴露了他们的数字足迹，例如社交媒体活动、私人照片和新闻报道，通常并未征得个人同意。 为了应对这一新兴威胁，我们提出了Protego，这是一种以用户为中心的隐私保护方法，旨在保护面部图像免受此类基于检索的隐私侵犯。Protego将用户的3D面部特征封装成一个与姿势无关的2D表示，并将其动态变形为一个自然外观的3D面具，该面具根据任何面部图像的姿态和表情定制并应用于在线分享之前。 鉴于现有方法的一个关键限制，Protego增强了面部识别模型的敏感性，使得即使在保护后的图像之间也无法进行匹配。实验表明，Protego显著降低了黑盒面部识别模型的检索精度，并且至少比现有的方法表现好2倍。此外，它还提供了前所未有的视觉连贯性，在视频设置中尤其重要，因为在这类场景下，一致性和自然外观至关重要。 总的来说，Protego有助于对抗滥用面部识别技术进行大规模监控和未经授权的身份追踪的行为。
URL
https://arxiv.org/abs/2508.02034
PDF
-
Backdoor Attacks on Deep Learning Face Detection
2025-08-01 13:29:26Quentin Le Roux, Yannick Teglia, Teddy Furon, Philippe Loubet-MoundiAbstract
Face Recognition Systems that operate in unconstrained environments capture images under varying conditions,such as inconsistent lighting, or diverse face poses. These challenges require including a Face Detection module that regresses bounding boxes and landmark coordinates for proper Face Alignment. This paper shows the effectiveness of Object Generation Attacks on Face Detection, dubbed Face Generation Attacks, and demonstrates for the first time a Landmark Shift Attack that backdoors the coordinate regression task performed by face detectors. We then offer mitigations against these vulnerabilities.
Abstract (translated)
在非约束环境中运行的面部识别系统会在不同条件下捕捉图像，例如光照不一致或不同的面部姿态。这些挑战需要包括一个面部检测模块来回归边界框和地标坐标以实现正确的面部对齐。本文展示了针对面部检测系统的物体生成攻击（称为面部生成攻击）的有效性，并首次演示了一种地标偏移攻击，这种攻击会篡改面部探测器执行的坐标回归任务。最后，我们提供了针对这些漏洞的缓解措施。
URL
https://arxiv.org/abs/2508.00620
PDF
-
Latent Diffusion Based Face Enhancement under Degraded Conditions for Forensic Face Recognition
2025-07-31 14:28:12Hassan Ugail, Hamad Mansour Alawar, AbdulNasser Abbas Zehi, Ahmed Mohammad Alkendi, Ismail Lujain JaleelAbstract
Face recognition systems experience severe performance degradation when processing low-quality forensic evidence imagery. This paper presents an evaluation of latent diffusion-based enhancement for improving face recognition under forensically relevant degradations. Using a dataset of 3,000 individuals from LFW with 24,000 recognition attempts, we implement the Flux.1 Kontext Dev pipeline with Facezoom LoRA adaptation to test against seven degradation categories, including compression artefacts, blur effects, and noise contamination. Our approach demonstrates substantial improvements, increasing overall recognition accuracy from 29.1% to 84.5% (55.4 percentage point improvement, 95% CI: [54.1, 56.7]). Statistical analysis reveals significant performance gains across all degradation types, with effect sizes exceeding conventional thresholds for practical significance. These findings establish the potential of sophisticated diffusion based enhancement in forensic face recognition applications.
Abstract (translated)
面部识别系统在处理低质量的法医证据图像时性能会严重下降。本文介绍了一种基于潜扩散增强技术来改善法医学相关退化条件下面部识别精度的评估方法。我们使用包含3000名来自LFW数据集个体和24000次面部识别尝试的数据，实现了带有Facezoom LoRA适配器的Flux.1 Kontext Dev流水线，并针对包括压缩伪影、模糊效果和噪声污染在内的七种退化类型进行了测试。我们的方法展示了显著改进，在所有降级条件下，总体识别精度从29.1%提高到84.5%（提升了55.4个百分点，95%的置信区间为[54.1, 56.7]）。统计分析显示在各种退化类型下均存在显着性能提升，效应大小超过了实用意义的传统阈值。这些发现确立了复杂扩散增强技术在法医面部识别应用中的潜力。
URL
https://arxiv.org/abs/2508.00941
PDF
-
Robust Deepfake Detection for Electronic Know Your Customer Systems Using Registered Images
2025-07-30 12:16:27Takuma Amada, Kazuya Kakizaki, Taiki Miyagawa, Akinori F. Ebihara, Kaede Shiohara, Toshihiko YamasakiAbstract
In this paper, we present a deepfake detection algorithm specifically designed for electronic Know Your Customer (eKYC) systems. To ensure the reliability of eKYC systems against deepfake attacks, it is essential to develop a robust deepfake detector capable of identifying both face swapping and face reenactment, while also being robust to image degradation. We address these challenges through three key contributions: (1)~Our approach evaluates the video's authenticity by detecting temporal inconsistencies in identity vectors extracted by face recognition models, leading to comprehensive detection of both face swapping and face reenactment. (2)~In addition to processing video input, the algorithm utilizes a registered image (assumed to be genuine) to calculate identity discrepancies between the input video and the registered image, significantly improving detection accuracy. (3)~We find that employing a face feature extractor trained on a larger dataset enhances both detection performance and robustness against image degradation. Our experimental results show that our proposed method accurately detects both face swapping and face reenactment comprehensively and is robust against various forms of unseen image degradation. Our source code is publicly available this https URL.
Abstract (translated)
在这篇论文中，我们提出了一种专为电子了解您的客户(eKYC)系统设计的深度伪造检测算法。为了确保eKYC系统能够抵御深度伪造攻击，开发一种既能识别面部替换也能识别面部再现，并且在图像退化情况下依然稳健的深度伪造检测器至关重要。通过以下三个关键贡献，我们解决了这些挑战： 1. 我们的方法通过对由面部识别模型提取的身份向量进行时间不一致性的检测来评估视频的真实性，从而全面地检测出面部替换和面部再现。 2. 除了处理视频输入之外，该算法还利用一个注册图像（假设为真实），计算输入视频与注册图像之间身份差异，显著提高了检测准确性。 3. 我们发现使用在更大数据集上训练的面部特征提取器可以提高检测性能，并增强其对各种形式图像退化的鲁棒性。 我们的实验结果表明，我们提出的方法能够准确且全面地检测出面部替换和面部再现，并且对于各种未见过的图像降质情况具有较强的鲁棒性。我们的源代码可在[此处](https://this https URL)公开获取。
URL
https://arxiv.org/abs/2507.22601
PDF
-
FaceGCD: Generalized Face Discovery via Dynamic Prefix Generation
2025-07-30 03:34:37Yunseok Oh, Dong-Wan ChoiAbstract
Recognizing and differentiating among both familiar and unfamiliar faces is a critical capability for face recognition systems and a key step toward artificial general intelligence (AGI). Motivated by this ability, this paper introduces generalized face discovery (GFD), a novel open-world face recognition task that unifies traditional face identification with generalized category discovery (GCD). GFD requires recognizing both labeled and unlabeled known identities (IDs) while simultaneously discovering new, previously unseen IDs. Unlike typical GCD settings, GFD poses unique challenges due to the high cardinality and fine-grained nature of face IDs, rendering existing GCD approaches ineffective. To tackle this problem, we propose FaceGCD, a method that dynamically constructs instance-specific feature extractors using lightweight, layer-wise prefixes. These prefixes are generated on the fly by a HyperNetwork, which adaptively outputs a set of prefix generators conditioned on each input image. This dynamic design enables FaceGCD to capture subtle identity-specific cues without relying on high-capacity static models. Extensive experiments demonstrate that FaceGCD significantly outperforms existing GCD methods and a strong face recognition baseline, ArcFace, achieving state-of-the-art results on the GFD task and advancing toward open-world face recognition.
Abstract (translated)
识别和区分熟悉与不熟悉的面孔是面部识别系统的一项关键能力，也是迈向人工通用智能（AGI）的重要一步。受到这一能力的启发，本文介绍了广义人脸发现（GFD），这是一种将传统的人脸识别与广义类别发现（GCD）相结合的新颖开放世界面部识别任务。GFD要求同时识别已标记和未标记的身份（ID），并在此过程中发现以前从未见过的新身份。与典型的GCD设置不同，由于面部ID具有高基数性和细粒度特性，因此现有的GCD方法对此类问题无效。 为了解决这一挑战，我们提出了一种名为FaceGCD的方法，该方法利用轻量级、逐层前缀动态构建实例特定的特征提取器。这些前缀由超网络实时生成，并根据每个输入图像条件化地输出一组前缀生成器。这种动态设计使FaceGCD能够在不依赖于高容量静态模型的情况下捕捉细微的身份特定线索。 广泛的实验表明，FaceGCD在现有GCD方法和强大的面部识别基线ArcFace上取得了显著的性能提升，在GFD任务中实现了最先进的结果，并为开放世界的面部识别技术进步作出了贡献。
URL
https://arxiv.org/abs/2507.22353
PDF
-
Investigation of Accuracy and Bias in Face Recognition Trained with Synthetic Data
2025-07-28 12:52:23Pavel Korshunov, Ketan Kotwal, Christophe Ecabert, Vidit Vidit, Amir Mohammadi, Sebastien MarcelAbstract
Synthetic data has emerged as a promising alternative for training face recognition (FR) models, offering advantages in scalability, privacy compliance, and potential for bias mitigation. However, critical questions remain on whether both high accuracy and fairness can be achieved with synthetic data. In this work, we evaluate the impact of synthetic data on bias and performance of FR systems. We generate balanced face dataset, FairFaceGen, using two state of the art text-to-image generators, Flux.1-dev and Stable Diffusion v3.5 (SD35), and combine them with several identity augmentation methods, including Arc2Face and four IP-Adapters. By maintaining equal identity count across synthetic and real datasets, we ensure fair comparisons when evaluating FR performance on standard (LFW, AgeDB-30, etc.) and challenging IJB-B/C benchmarks and FR bias on Racial Faces in-the-Wild (RFW) dataset. Our results demonstrate that although synthetic data still lags behind the real datasets in the generalization on IJB-B/C, demographically balanced synthetic datasets, especially those generated with SD35, show potential for bias mitigation. We also observe that the number and quality of intra-class augmentations significantly affect FR accuracy and fairness. These findings provide practical guidelines for constructing fairer FR systems using synthetic data.
Abstract (translated)
合成数据已成为训练面部识别（FR）模型的有前途的替代方案，它在可扩展性、隐私合规性和潜在的偏见缓解方面具有优势。然而，关于是否可以通过合成数据同时实现高准确率和公平性的关键问题仍然存在。在这项工作中，我们评估了合成数据对面部识别系统中的偏差和性能的影响。我们使用两个最先进的文本到图像生成器（Flux.1-dev 和 Stable Diffusion v3.5 (SD35)）生成了一个平衡的面部数据集FairFaceGen，并结合了几种身份增强方法，包括Arc2Face和四个IP-Adapters。通过在合成数据集和真实数据集中保持相同的身份计数，我们在评估标准（LFW、AgeDB-30等）和具有挑战性的IJB-B/C基准测试以及使用RFW数据集进行的面部识别偏见时确保了公平比较。 我们的结果表明，尽管在IJB-B/C上的泛化能力方面合成数据仍落后于真实数据集，但人口统计学平衡的合成数据集（尤其是用SD35生成的数据集）在减轻偏见方面显示出潜力。我们还观察到，类内增强的数量和质量显著影响面部识别的准确率和公平性。这些发现为使用合成数据构建更公正的面部识别系统提供了实用指南。
URL
https://arxiv.org/abs/2507.20782
PDF
-
Reinforced Embodied Active Defense: Exploiting Adaptive Interaction for Robust Visual Perception in Adversarial 3D Environments
2025-07-24 14:56:21Xiao Yang, Lingxuan Wu, Lizhong Wang, Chengyang Ying, Hang Su, Jun ZhuAbstract
Adversarial attacks in 3D environments have emerged as a critical threat to the reliability of visual perception systems, particularly in safety-sensitive applications such as identity verification and autonomous driving. These attacks employ adversarial patches and 3D objects to manipulate deep neural network (DNN) predictions by exploiting vulnerabilities within complex scenes. Existing defense mechanisms, such as adversarial training and purification, primarily employ passive strategies to enhance robustness. However, these approaches often rely on pre-defined assumptions about adversarial tactics, limiting their adaptability in dynamic 3D settings. To address these challenges, we introduce Reinforced Embodied Active Defense (Rein-EAD), a proactive defense framework that leverages adaptive exploration and interaction with the environment to improve perception robustness in 3D adversarial contexts. By implementing a multi-step objective that balances immediate prediction accuracy with predictive entropy minimization, Rein-EAD optimizes defense strategies over a multi-step horizon. Additionally, Rein-EAD involves an uncertainty-oriented reward-shaping mechanism that facilitates efficient policy updates, thereby reducing computational overhead and supporting real-world applicability without the need for differentiable environments. Comprehensive experiments validate the effectiveness of Rein-EAD, demonstrating a substantial reduction in attack success rates while preserving standard accuracy across diverse tasks. Notably, Rein-EAD exhibits robust generalization to unseen and adaptive attacks, making it suitable for real-world complex tasks, including 3D object classification, face recognition and autonomous driving.
Abstract (translated)
在三维环境中，对抗性攻击已成为视觉感知系统可靠性的关键威胁，特别是在身份验证和自动驾驶等安全敏感应用中。这些攻击利用对抗性贴纸和3D物体来操纵深度神经网络（DNN）的预测，并通过利用复杂场景中的漏洞进行操作。现有的防御机制，如对抗训练和净化，主要采用被动策略来增强系统的鲁棒性。然而，这些方法往往依赖于对敌手战术的预定义假设，从而限制了它们在动态3D环境中的适应能力。 为了解决这些问题，我们引入了一种名为强化嵌入式主动防御（Reinforced Embodied Active Defense, Rein-EAD）的框架。该框架采用自适应探索和与环境互动的方式，以提高三维对抗场景下的感知鲁棒性。通过实施一个多步目标，平衡即时预测准确性和预测熵最小化，Rein-EAD可以在多步规划范围内优化防御策略。此外，Rein-EAD涉及一种不确定性导向的奖励塑造机制，促进了高效的政策更新，从而减少了计算开销，并支持在非可微环境中实现现实世界的适用性。 综合实验验证了Rein-EAD的有效性，显示其能够在保持标准准确性的前提下显著降低攻击成功率。值得注意的是，Rein-EAD表现出对未见过和适应性攻击的鲁棒泛化能力，使其适用于包括3D物体分类、面部识别及自动驾驶在内的现实世界复杂任务。
URL
https://arxiv.org/abs/2507.18484
PDF
-
Autonomous UAV Navigation for Search and Rescue Missions Using Computer Vision and Convolutional Neural Networks
2025-07-24 07:54:45Luka \v{S}iktar, Branimir \'Caran, Bojan \v{S}ekoranja, Marko \v{S}vacoAbstract
In this paper, we present a subsystem, using Unmanned Aerial Vehicles (UAV), for search and rescue missions, focusing on people detection, face recognition and tracking of identified individuals. The proposed solution integrates a UAV with ROS2 framework, that utilizes multiple convolutional neural networks (CNN) for search missions. System identification and PD controller deployment are performed for autonomous UAV navigation. The ROS2 environment utilizes the YOLOv11 and YOLOv11-pose CNNs for tracking purposes, and the dlib library CNN for face recognition. The system detects a specific individual, performs face recognition and starts tracking. If the individual is not yet known, the UAV operator can manually locate the person, save their facial image and immediately initiate the tracking process. The tracking process relies on specific keypoints identified on the human body using the YOLOv11-pose CNN model. These keypoints are used to track a specific individual and maintain a safe distance. To enhance accurate tracking, system identification is performed, based on measurement data from the UAVs IMU. The identified system parameters are used to design PD controllers that utilize YOLOv11-pose to estimate the distance between the UAVs camera and the identified individual. The initial experiments, conducted on 14 known individuals, demonstrated that the proposed subsystem can be successfully used in real time. The next step involves implementing the system on a large experimental UAV for field use and integrating autonomous navigation with GPS-guided control for rescue operations planning.
Abstract (translated)
在这篇论文中，我们提出了一种使用无人驾驶航空器（UAV）的子系统，用于搜索和救援任务，重点在于人员检测、面部识别以及对已识别个人进行追踪。该解决方案将UAV与ROS2框架相结合，并利用多个卷积神经网络（CNN）执行搜索任务。为了实现自主飞行导航，进行了系统辨识并部署了PD控制器。在ROS2环境中，使用YOLOv11和YOLOv11-pose CNN来执行追踪任务，而dlib库的CNN用于面部识别。该系统能够检测特定个体、进行面部识别，并开始跟踪。如果尚未认识该个体，UAV操作员可以手动定位此人，保存其面部图像并立即启动追踪过程。追踪过程中，使用YOLOv11-pose CNN模型在人体上确定的关键点来追踪特定个人并保持安全距离。为了提高精确的追踪效果，基于UAV惯性测量单元（IMU）提供的测量数据进行了系统辨识，并利用该参数设计了PD控制器以通过YOLOv11-pose估算无人机摄像头与识别个体之间的距离。在初始实验中，我们对14名已知人员进行了测试，证明所提出的子系统可以实现实时应用。下一步计划是将该系统集成到一个大型试验用UAV上进行实地使用，并结合GPS引导控制以实现救援操作的规划。
URL
https://arxiv.org/abs/2507.18160
PDF
-
A Comprehensive Evaluation Framework for the Study of the Effects of Facial Filters on Face Recognition Accuracy
2025-07-23 17:43:35Kagan Ozturk, Louisa Conwill, Jacob Gutierrez, Kevin Bowyer, Walter J. ScheirerAbstract
Facial filters are now commonplace for social media users around the world. Previous work has demonstrated that facial filters can negatively impact automated face recognition performance. However, these studies focus on small numbers of hand-picked filters in particular styles. In order to more effectively incorporate the wide ranges of filters present on various social media applications, we introduce a framework that allows for larger-scale study of the impact of facial filters on automated recognition. This framework includes a controlled dataset of face images, a principled filter selection process that selects a representative range of filters for experimentation, and a set of experiments to evaluate the filters' impact on recognition. We demonstrate our framework with a case study of filters from the American applications Instagram and Snapchat and the Chinese applications Meitu and Pitu to uncover cross-cultural differences. Finally, we show how the filtering effect in a face embedding space can easily be detected and restored to improve face recognition performance.
Abstract (translated)
面部滤镜现在已成为全球社交媒体用户的一种常见工具。先前的研究表明，面部滤镜可能对自动化面部识别的性能产生负面影响。然而，这些研究主要集中在少数特定风格的手选滤镜上。为了更有效地应对各种社交媒体应用中广泛存在的滤镜问题，我们引入了一个框架，该框架允许进行更大规模地研究面部滤镜对自动识别的影响。此框架包括一个受控的脸部图像数据集、一种有原则的滤镜选择过程（用于为实验选取具有代表性的滤镜范围），以及一系列实验来评估这些滤镜对面部识别性能的影响。 我们通过分析来自美国应用Instagram和Snapchat，以及中国应用美图秀秀(Meitu)和皮土(Pitu)的滤镜进行案例研究，以揭示跨文化差异。最后，我们展示了如何在面部嵌入空间中检测并修复滤镜效果，从而改善面部识别性能。
URL
https://arxiv.org/abs/2507.17729
PDF
-
Vec2Face+ for Face Dataset Generation
2025-07-23 04:34:56Haiyu Wu, Jaskirat Singh, Sicong Tian, Liang Zheng, Kevin W. BowyerAbstract
When synthesizing identities as face recognition training data, it is generally believed that large inter-class separability and intra-class attribute variation are essential for synthesizing a quality dataset. % This belief is generally correct, and this is what we aim for. However, when increasing intra-class variation, existing methods overlook the necessity of maintaining intra-class identity consistency. % To address this and generate high-quality face training data, we propose Vec2Face+, a generative model that creates images directly from image features and allows for continuous and easy control of face identities and attributes. Using Vec2Face+, we obtain datasets with proper inter-class separability and intra-class variation and identity consistency using three strategies: 1) we sample vectors sufficiently different from others to generate well-separated identities; 2) we propose an AttrOP algorithm for increasing general attribute variations; 3) we propose LoRA-based pose control for generating images with profile head poses, which is more efficient and identity-preserving than AttrOP. % Our system generates VFace10K, a synthetic face dataset with 10K identities, which allows an FR model to achieve state-of-the-art accuracy on seven real-world test sets. Scaling the size to 4M and 12M images, the corresponding VFace100K and VFace300K datasets yield higher accuracy than the real-world training dataset, CASIA-WebFace, on five real-world test sets. This is the first time a synthetic dataset beats the CASIA-WebFace in average accuracy. In addition, we find that only 1 out of 11 synthetic datasets outperforms random guessing (\emph{i.e., 50\%}) in twin verification and that models trained with synthetic identities are more biased than those trained with real identities. Both are important aspects for future investigation.
Abstract (translated)
在将身份合成用作面部识别训练数据时，通常认为大类间分离性和类内属性变化对于生成高质量的数据集至关重要。这种看法通常是正确的，这也是我们的目标所在。然而，在增加类内变化的同时，现有方法往往忽略了保持类内身份一致性的重要性。为了应对这一挑战并生成高质量的面部训练数据，我们提出了一种名为Vec2Face+的生成模型，该模型可以直接从图像特征创建图像，并允许对人脸身份和属性进行连续且易于控制的操作。通过使用Vec2Face+，我们采用三种策略获得了具备适当类间分离性和类内变化及一致性身份的数据集：1）采样足够不同于其他样本的向量以生成具有较好隔离性的身份；2）提出了一种AttrOP算法来增加一般属性的变化；3）提出了基于LoRA的姿态控制方法，用于生成带有侧脸姿态的图像，这种方法比AttrOP更高效且更能保持身份一致性。 我们的系统产生了VFace10K，这是一个拥有10,000个不同身份的人工合成面部数据集，在七个真实世界的测试集中使面部识别模型达到了最先进的精度水平。当我们扩展该数据集规模至4M和12M图像时，对应的VFace100K和VFace300K数据集在五个实际测试集合上比现实世界训练数据集CASIA-WebFace的平均准确度更高。这是首次合成数据集在平均精度上超越了CASIA-WebFace。 此外，我们发现只有11个合成数据集中的一项超过了随机猜测（即50%）的表现，在双胞胎验证任务中。另外，使用合成身份训练出的模型比使用真实身份训练出的模型更加偏颇。这两者都是未来研究的重要方面。
URL
https://arxiv.org/abs/2507.17192
PDF
-
Enhancing Domain Diversity in Synthetic Data Face Recognition with Dataset Fusion
2025-07-22 17:36:48Anjith George, Sebastien MarcelAbstract
While the accuracy of face recognition systems has improved significantly in recent years, the datasets used to train these models are often collected through web crawling without the explicit consent of users, raising ethical and privacy concerns. To address this, many recent approaches have explored the use of synthetic data for training face recognition models. However, these models typically underperform compared to those trained on real-world data. A common limitation is that a single generator model is often used to create the entire synthetic dataset, leading to model-specific artifacts that may cause overfitting to the generator's inherent biases and artifacts. In this work, we propose a solution by combining two state-of-the-art synthetic face datasets generated using architecturally distinct backbones. This fusion reduces model-specific artifacts, enhances diversity in pose, lighting, and demographics, and implicitly regularizes the face recognition model by emphasizing identity-relevant features. We evaluate the performance of models trained on this combined dataset using standard face recognition benchmarks and demonstrate that our approach achieves superior performance across many of these benchmarks.
Abstract (translated)
虽然近年来面部识别系统的准确性有了显著提高，但用于训练这些模型的数据集通常是通过网络爬虫收集的，并未获得用户的明确同意，这引发了伦理和隐私方面的担忧。为了解决这个问题，许多近期的方法探索了使用合成数据来训练面部识别模型的可能性。然而，与基于真实世界数据训练的模型相比，这类方法通常表现较差。常见的限制是单个生成器模型通常用于创建整个合成数据集，导致特定于该模型的偏差和伪影，可能导致过度拟合到生成器本身的偏见和伪影上。 在本研究中，我们提出了一种解决方案，即结合使用两种最新的合成面部数据集，这些数据集是由架构不同的骨干网络生成的。这种融合减少了特定于模型的伪影，增强了姿态、光照及人口统计学方面的多样性，并通过强调与身份相关的特征隐式地对面部识别模型进行了正则化处理。 我们评估了在标准面部识别基准测试中使用该组合数据集训练的模型性能，并证明我们的方法在此类基准中的表现明显优于其他方法。
URL
https://arxiv.org/abs/2507.16790
PDF
