Face_Recognition
-
Masked Face Recognition under Different Backbones
2026-01-23 04:33:37Bo Zhang, Ming Zhang, Kun Wu, Lei Bian, Yi LinAbstract
Erratum to the paper (Zhang et al., 2025): corrections to Table IV and the data in Page 3, Section A. In the post-pandemic era, a high proportion of civil aviation passengers wear masks during security checks, posing significant challenges to traditional face recognition models. The backbone network serves as the core component of face recognition models. In standard tests, r100 series models excelled (98%+ accuracy at 0.01% FAR in face comparison, high top1/top5 in search). r50 ranked second, r34_mask_v1 lagged. In masked tests, r100_mask_v2 led (90.07% accuracy), r50_mask_v3 performed best among r50 but trailed r100. Vit-Small/Tiny showed strong masked performance with gains in effectiveness. Through extensive comparative experiments, this paper conducts a comprehensive evaluation of several core backbone networks, aiming to reveal the impacts of different models on face recognition with and without masks, and provide specific deployment recommendations.
Abstract (translated)
URL
https://arxiv.org/abs/2601.16440
PDF
-
Evaluating Multimodal Large Language Models for Heterogeneous Face Recognition
2026-01-21 19:17:21Hatef Otroshi Shahreza, Anjith George, S\'ebastien MarcelAbstract
Multimodal Large Language Models (MLLMs) have recently demonstrated strong performance on a wide range of vision-language tasks, raising interest in their potential use for biometric applications. In this paper, we conduct a systematic evaluation of state-of-the-art MLLMs for heterogeneous face recognition (HFR), where enrollment and probe images are from different sensing modalities, including visual (VIS), near infrared (NIR), short-wave infrared (SWIR), and thermal camera. We benchmark multiple open-source MLLMs across several cross-modality scenarios, including VIS-NIR, VIS-SWIR, and VIS-THERMAL face recognition. The recognition performance of MLLMs is evaluated using biometric protocols and based on different metrics, including Acquire Rate, Equal Error Rate (EER), and True Accept Rate (TAR). Our results reveal substantial performance gaps between MLLMs and classical face recognition systems, particularly under challenging cross-spectral conditions, in spite of recent advances in MLLMs. Our findings highlight the limitations of current MLLMs for HFR and also the importance of rigorous biometric evaluation when considering their deployment in face recognition systems.
Abstract (translated)
最近，多模态大型语言模型（MLLMs）在广泛的视觉-语言任务中表现出色，引发了人们对其在生物识别应用中的潜在用途的兴趣。在这篇论文中，我们对最先进的MLLMs进行了系统评估，用于异构人脸识别（HFR），其中注册和探针图像来自不同的传感模式，包括可见光（VIS）、近红外（NIR）、短波红外（SWIR）和热成像相机。我们在多个跨模态场景下对多种开源MLLM进行基准测试，其中包括VIS-NIR、VIS-SWIR和VIS-THERMAL人脸识别。 我们使用生物识别协议并基于不同的指标，包括获取率(Acquire Rate)、等错误率(EER)和真正接受率(TAR)，评估了MLLMs的识别性能。我们的研究结果揭示了在具有挑战性的跨光谱条件下，即使是最近取得的进步，MLLMs与经典的人脸识别系统之间仍然存在显著的性能差距。这些发现强调了当前MLLMs在HFR中的局限性，并且也突显了在其部署于人脸识别系统时进行严格的生物特征评估的重要性。
URL
https://arxiv.org/abs/2601.15406
PDF
-
Single-Pixel Vision-Language Model for Intrinsic Privacy-Preserving Behavioral Intelligence
2026-01-21 09:11:26Hongjun An, Yiliang Song, Jiawei Shao, Zhe Sun, Xuelong LiAbstract
Adverse social interactions, such as bullying, harassment, and other illicit activities, pose significant threats to individual well-being and public safety, leaving profound impacts on physical and mental health. However, these critical events frequently occur in privacy-sensitive environments like restrooms, and changing rooms, where conventional surveillance is prohibited or severely restricted by stringent privacy regulations and ethical concerns. Here, we propose the Single-Pixel Vision-Language Model (SP-VLM), a novel framework that reimagines secure environmental monitoring. It achieves intrinsic privacy-by-design by capturing human dynamics through inherently low-dimensional single-pixel modalities and inferring complex behavioral patterns via seamless vision-language integration. Building on this framework, we demonstrate that single-pixel sensing intrinsically suppresses identity recoverability, rendering state-of-the-art face recognition systems ineffective below a critical sampling rate. We further show that SP-VLM can nonetheless extract meaningful behavioral semantics, enabling robust anomaly detection, people counting, and activity understanding from severely degraded single-pixel observations. Combining these findings, we identify a practical sampling-rate regime in which behavioral intelligence emerges while personal identity remains strongly protected. Together, these results point to a human-rights-aligned pathway for safety monitoring that can support timely intervention without normalizing intrusive surveillance in privacy-sensitive spaces.
Abstract (translated)
URL
https://arxiv.org/abs/2601.17050
PDF
-
Real-Time Multi-Modal Embedded Vision Framework for Object Detection Facial Emotion Recognition and Biometric Identification on Low-Power Edge Platforms
2026-01-17 09:06:47S. M. Khalid Bin Zahid, Md. Rakibul Hasan Nishat, Abdul Hasib, Md. Rakibul Hasan, Md. Ashiqussalehin, Md. Sahadat Hossen Sajib, A. S. M. Ahsanul Sarkar AkibAbstract
Intelligent surveillance systems often handle perceptual tasks such as object detection, facial recognition, and emotion analysis independently, but they lack a unified, adaptive runtime scheduler that dynamically allocates computational resources based on contextual triggers. This limits their holistic understanding and efficiency on low-power edge devices. To address this, we present a real-time multi-modal vision framework that integrates object detection, owner-specific face recognition, and emotion detection into a unified pipeline deployed on a Raspberry Pi 5 edge platform. The core of our system is an adaptive scheduling mechanism that reduces computational load by 65\% compared to continuous processing by selectively activating modules such as, YOLOv8n for object detection, a custom FaceNet-based embedding system for facial recognition, and DeepFace's CNN for emotion classification. Experimental results demonstrate the system's efficacy, with the object detection module achieving an Average Precision (AP) of 0.861, facial recognition attaining 88\% accuracy, and emotion detection showing strong discriminatory power (AUC up to 0.97 for specific emotions), while operating at 5.6 frames per second. Our work demonstrates that context-aware scheduling is the key to unlocking complex multi-modal AI on cost-effective edge hardware, making intelligent perception more accessible and privacy-preserving.
Abstract (translated)
智能监控系统通常独立处理如物体检测、面部识别和情绪分析等感知任务，但它们缺乏一个统一的自适应运行时调度器，该调度器可以根据上下文触发动态分配计算资源。这限制了其在低功耗边缘设备上的整体理解和效率。为解决这一问题，我们提出了一种实时多模态视觉框架，该框架将物体检测、基于个人的脸部识别和情绪检测整合到一个统一的管道中，并部署在Raspberry Pi 5边缘平台上。 我们系统的核心是一个自适应调度机制，通过选择性激活模块（例如用于对象检测的YOLOv8n、用于面部识别的定制FaceNet嵌入系统以及用于情绪分类的DeepFace卷积神经网络）来减少连续处理所需的计算负载达65%。实验结果显示，该系统的有效性得到了证明：物体检测模块实现了0.861的平均精度（AP），面部识别达到了88%的准确率，而情绪检测对于特定情感显示出强大的区分能力（AUC高达0.97）。同时，在每秒处理5.6帧的情况下，系统运行稳定。 我们的工作表明，基于上下文感知的调度是解锁复杂多模态AI的关键，并在成本效益高的边缘硬件上实现智能感知，使之更加普及且具有隐私保护功能。
URL
https://arxiv.org/abs/2601.11970
PDF
-
SPOT-Face: Forensic Face Identification using Attention Guided Optimal Transport
2026-01-14 07:02:21Ravi Shankar Prasad, Dinesh SinghAbstract
Person identification in forensic investigations becomes very challenging when common identification means for DNA (i.e., hair strands, soft tissue) are not available. Current methods utilize deep learning methods for face recognition. However, these methods lack effective mechanisms to model cross-domain structural correspondence between two different forensic modalities. In this paper, we introduce a SPOT-Face, a superpixel graph-based framework designed for cross-domain forensic face identification of victims using their skeleton and sketch images. Our unified framework involves constructing a superpixel-based graph from an image and then using different graph neural networks(GNNs) backbones to extract the embeddings of these graphs, while cross-domain correspondence is established through attention-guided optimal transport mechanism. We have evaluated our proposed framework on two publicly available dataset: IIT\_Mandi\_S2F (S2F) and CUFS. Extensive experiments were conducted to evaluate our proposed framework. The experimental results show significant improvement in identification metrics ( i.e., Recall, mAP) over existing graph-based baselines. Furthermore, our framework demonstrates to be highly effective for matching skulls and sketches to faces in forensic investigations.
Abstract (translated)
在法医调查中，当常见的DNA鉴定手段（如毛发、软组织）不可用时，人员识别变得非常具有挑战性。目前的方法主要依赖于深度学习技术进行人脸识别，但这些方法缺乏有效的机制来建模两种不同法医模式之间的跨域结构对应关系。本文介绍了一种名为SPOT-Face的框架，该框架基于超像素图，旨在使用受害者的骨骼和素描图像来进行跨域法医面部识别。 我们的统一框架包括从一张图片中构建一个基于超像素的图，并利用不同的图神经网络（GNN）骨干来提取这些图的嵌入，同时通过注意力引导的最佳传输机制建立跨域对应关系。我们在两个公开的数据集上评估了我们提出的框架：IIT_Mandi_S2F (S2F) 和 CUFS。 进行了广泛的实验以评估我们的提议框架。实验证明，在识别指标（如召回率、mAP）方面，与现有的基于图的基线相比，有了显著改进。此外，我们的框架还表现出强大的能力，能够将颅骨和素描匹配到面部图像中，这对于法医调查来说非常有效。 该研究提供了一种创新的方法来解决跨模态法医识别中的挑战，特别是在缺乏DNA样本的情况下，为受害者身份的确定提供了新的可能性。
URL
https://arxiv.org/abs/2601.09229
PDF
-
One-Shot Identification with Different Neural Network Approaches
2026-01-13 07:13:31Janis Mohr, J\"org FrochteAbstract
Convolutional neural networks (CNNs) have been widely used in the computer vision community, significantly improving the state-of-the-art. But learning good features often is computationally expensive in machine learning settings and is especially difficult when there is a lack of data. One-shot learning is one such area where only limited data is available. In one-shot learning, predictions have to be made after seeing only one example from one class, which requires special techniques. In this paper we explore different approaches to one-shot identification tasks in different domains including an industrial application and face recognition. We use a special technique with stacked images and use siamese capsule networks. It is encouraging to see that the approach using capsule architecture achieves strong results and exceeds other techniques on a wide range of datasets from industrial application to face recognition benchmarks while being easy to use and optimise.
Abstract (translated)
卷积神经网络（CNNs）在计算机视觉社区中得到了广泛应用，显著提升了技术前沿。然而，在机器学习环境中学习良好的特征通常需要较高的计算成本，尤其是在数据不足的情况下尤其困难。一次性学习就是这样一个领域，仅提供有限的数据。在一次性学习中，模型必须基于看到的一个类别的单一示例进行预测，这要求使用特殊的技术。 本文探讨了不同的一次性识别任务方法，涵盖工业应用和人脸识别等不同领域。我们采用了一种特殊的叠图技术，并利用孪生胶囊网络（Siamese Capsule Networks）来解决这些问题。令人鼓舞的是，基于胶囊架构的方法在从工业应用到人脸识别基准测试的广泛数据集上取得了优异的结果，并且易于使用和优化，超过了其他方法的表现。
URL
https://arxiv.org/abs/2601.08278
PDF
-
A survey of facial recognition techniques
2026-01-09 16:24:44Aya Kaysan BahjatAbstract
As multimedia content is quickly growing, the field of facial recognition has become one of the major research fields, particularly in the recent years. The most problematic area to researchers in image processing and computer vision is the human face which is a complex object with myriads of distinctive features that can be used to identify the face. The survey of this survey is particularly focused on most challenging facial characteristics, including differences in the light, ageing, variation in poses, partial occlusion, and facial expression and presents methodological solutions. The factors, therefore, are inevitable in the creation of effective facial recognition mechanisms used on facial images. This paper reviews the most sophisticated methods of facial detection which are Hidden Markov Models, Principal Component Analysis (PCA), Elastic Cluster Plot Matching, Support Vector Machine (SVM), Gabor Waves, Artificial Neural Networks (ANN), Eigenfaces, Independent Component Analysis (ICA), and 3D Morphable Model. Alongside the works mentioned above, we have also analyzed the images of a number of facial databases, namely JAFEE, FEI, Yale, LFW, AT&T (then called ORL), and AR (created by Martinez and Benavente), to analyze the results. However, this survey is aimed at giving a thorough literature review of face recognition, and its applications, and some experimental results are provided at the end after a detailed discussion.
Abstract (translated)
随着多媒体内容的迅速增长，面部识别领域已成为近年来研究的重点之一。对于图像处理和计算机视觉的研究人员而言，最具挑战性的问题是人类脸部这个复杂对象，它包含众多独特的特征可以用来识别人脸。本文综述特别关注最具有挑战性的面部特性，包括光照变化、老化过程、姿势的变化、部分遮挡以及表情的影响，并提供了方法论上的解决方案。这些因素在创造有效的面部识别机制时不可避免。 本文回顾了最先进的面部检测方法，其中包括隐马尔可夫模型（Hidden Markov Models）、主成分分析法（PCA）、弹性集群图匹配（Elastic Cluster Plot Matching）、支持向量机（SVM）、Gabor滤波器、人工神经网络（ANN）、特征脸技术（Eigenfaces）、独立成分分析（ICA）以及3D形态学模型。除了上述工作之外，我们还对多个面部数据库的图像进行了分析，包括JAFEE、FEI、Yale、LFW、AT&T（当时称ORL），以及Martinez和Benavente创建的AR数据库，以进一步分析这些方法的效果。 然而，本文的主要目的是提供全面的脸部识别文献综述及其应用，并在详细讨论之后提供了部分实验结果。
URL
https://arxiv.org/abs/2601.06239
PDF
-
ViTNT-FIQA: Training-Free Face Image Quality Assessment with Vision Transformers
2026-01-09 11:46:25Guray Ozgur, Eduarda Caldeira, Tahar Chettaoui, Jan Niklas Kolf, Marco Huber, Naser Damer, Fadi BoutrosAbstract
Face Image Quality Assessment (FIQA) is essential for reliable face recognition systems. Current approaches primarily exploit only final-layer representations, while training-free methods require multiple forward passes or backpropagation. We propose ViTNT-FIQA, a training-free approach that measures the stability of patch embedding evolution across intermediate Vision Transformer (ViT) blocks. We demonstrate that high-quality face images exhibit stable feature refinement trajectories across blocks, while degraded images show erratic transformations. Our method computes Euclidean distances between L2-normalized patch embeddings from consecutive transformer blocks and aggregates them into image-level quality scores. We empirically validate this correlation on a quality-labeled synthetic dataset with controlled degradation levels. Unlike existing training-free approaches, ViTNT-FIQA requires only a single forward pass without backpropagation or architectural modifications. Through extensive evaluation on eight benchmarks (LFW, AgeDB-30, CFP-FP, CALFW, Adience, CPLFW, XQLFW, IJB-C), we show that ViTNT-FIQA achieves competitive performance with state-of-the-art methods while maintaining computational efficiency and immediate applicability to any pre-trained ViT-based face recognition model.
Abstract (translated)
面部图像质量评估（FIQA）对于可靠的脸部识别系统至关重要。目前的方法主要利用最终层的表示，而无需训练的方法则需要多次前向传递或反向传播。我们提出了ViTNT-FIQA，这是一种无需训练的方法，它通过测量中间Vision Transformer (ViT) 块中补丁嵌入演化的稳定性来评估图像质量。我们展示了高质量的脸部图像在整个块中的特征细化路径表现出稳定的状态，而质量较差的图像则显示出不规则的变化。我们的方法计算了连续变换器块之间的L2标准化补丁嵌入的欧几里得距离，并将这些距离汇总为图像级别的质量分数。我们在一个合成数据集上验证了这种相关性，该数据集中存在受控降级水平的质量标记。 与现有的无需训练的方法不同，ViTNT-FIQA仅需要单次前向传递，不需要反向传播或架构修改。通过在八个基准测试（LFW、AgeDB-30、CFP-FP、CALFW、Adience、CPLFW、XQLFW、IJB-C）上的广泛评估，我们展示了ViTNT-FIQA能够实现与现有最佳方法相当的性能，同时保持计算效率，并且可以立即应用于任何基于预训练Vision Transformer的脸部识别模型。
URL
https://arxiv.org/abs/2601.05741
PDF
-
GaussianSwap: Animatable Video Face Swapping with 3D Gaussian Splatting
2026-01-09 03:39:29Xuan Cheng, Jiahao Rao, Chengyang Li, Wenhao Wang, Weilin Chen, Lvqing YangAbstract
We introduce GaussianSwap, a novel video face swapping framework that constructs a 3D Gaussian Splatting based face avatar from a target video while transferring identity from a source image to the avatar. Conventional video swapping frameworks are limited to generating facial representations in pixel-based formats. The resulting swapped faces exist merely as a set of unstructured pixels without any capacity for animation or interactive manipulation. Our work introduces a paradigm shift from conventional pixel-based video generation to the creation of high-fidelity avatar with swapped faces. The framework first preprocesses target video to extract FLAME parameters, camera poses and segmentation masks, and then rigs 3D Gaussian splats to the FLAME model across frames, enabling dynamic facial control. To ensure identity preserving, we propose an compound identity embedding constructed from three state-of-the-art face recognition models for avatar finetuning. Finally, we render the face-swapped avatar on the background frames to obtain the face-swapped video. Experimental results demonstrate that GaussianSwap achieves superior identity preservation, visual clarity and temporal consistency, while enabling previously unattainable interactive applications.
Abstract (translated)
我们介绍了一种新的视频面部交换框架——GaussianSwap，该框架通过从目标视频构建基于3D高斯点阵的脸部虚拟形象，并将源图像的身份信息转移到虚拟形象中来实现面部替换。传统的视频面部交换框架仅限于生成像素格式的面部表示形式，而置换后的脸部仅仅是无结构的一组像素集合，缺乏动画和互动操作的能力。 我们的工作打破了传统基于像素的视频生成模式，实现了高保真度、具备身份交换功能的虚拟形象创建。该框架首先对目标视频进行预处理以提取FLAME参数（面部模型）、相机姿态以及分割掩码，并在不同帧之间将3D高斯点阵绑定到FLAME模型上，从而实现动态面部控制。 为了确保身份一致性，我们提出了一种由三种最先进的面部识别模型组成的复合身份嵌入方法，用于虚拟形象的微调。最后，我们将面部交换后的虚拟形象渲染回背景帧中以生成最终的面部交换视频。 实验结果表明，GaussianSwap在身份保持、视觉清晰度和时间一致性方面表现出色，并且实现了以前难以实现的互动应用功能。
URL
https://arxiv.org/abs/2601.05511
PDF
-
Two-step Authentication: Multi-biometric System Using Voice and Facial Recognition
2026-01-09 02:11:50Kuan Wei Chen, Ting Yi Lin, Wen Ren Yang, Aryan Kesarwani, Riya SinghAbstract
We present a cost-effective two-step authentication system that integrates face identification and speaker verification using only a camera and microphone available on common devices. The pipeline first performs face recognition to identify a candidate user from a small enrolled group, then performs voice recognition only against the matched identity to reduce computation and improve robustness. For face recognition, a pruned VGG-16 based classifier is trained on an augmented dataset of 924 images from five subjects, with faces localized by MTCNN; it achieves 95.1% accuracy. For voice recognition, a CNN speaker-verification model trained on LibriSpeech (train-other-360) attains 98.9% accuracy and 3.456% EER on test-clean. Source code and trained models are available at this https URL.
Abstract (translated)
我们提出了一种成本效益高的两步认证系统，该系统仅使用常见设备上配备的摄像头和麦克风，集成了面部识别和说话人验证。该管道首先执行面部识别以从一个小规模注册用户组中确定候选用户，然后只针对匹配的身份进行语音识别，以此来减少计算量并提高鲁棒性。对于面部识别，我们对一个由五个主体提供的924张图片的增强数据集训练了一个基于修剪后的VGG-16分类器，并通过MTCNN定位人脸；其准确率为95.1%。对于语音识别，我们在LibriSpeech（train-other-360）上训练了一个卷积神经网络说话人验证模型，在测试集test-clean上的准确性为98.9%，等错误率(EER)为3.456%。源代码和预训练模型可在此链接获取：[提供实际的URL地址]。
URL
https://arxiv.org/abs/2601.06218
PDF
-
Mitigating Longitudinal Performance Degradation in Child Face Recognition Using Synthetic Data
2026-01-04 23:03:03Afzal Hossain, Stephanie SchuckersAbstract
Longitudinal face recognition in children remains challenging due to rapid and nonlinear facial growth, which causes template drift and increasing verification errors over time. This work investigates whether synthetic face data can act as a longitudinal stabilizer by improving temporal robustness of child face recognition models. Using an identity disjoint protocol on the Young Face Aging (YFA) dataset, we evaluate three settings: (i) pretrained MagFace embeddings without dataset specific fine-tuning, (ii) MagFace fine-tuned using authentic training faces only, and (iii) MagFace fine-tuned using a combination of authentic and synthetically generated training faces. Synthetic data is generated using StyleGAN2 ADA and incorporated exclusively within the training identities; a post generation filtering step is applied to mitigate identity leakage and remove artifact affected samples. Experimental results across enrollment verification gaps from 6 to 36 months show that synthetic-augmented fine tuning substantially reduces error rates relative to both the pretrained baseline and real only fine tuning. These findings provide a risk aware assessment of synthetic augmentation for improving identity persistence in pediatric face recognition.
Abstract (translated)
儿童纵向面部识别仍然面临挑战，因为快速且非线性的面部成长会导致模板漂移和随时间推移验证错误的增加。这项工作研究了合成面部数据是否可以通过提高儿童面部识别模型的时间鲁棒性来充当纵向稳定器。在Young Face Aging (YFA) 数据集上采用身份独立协议后，我们评估了三种设置：(i) 不经过特定数据集微调的预训练MagFace嵌入；(ii) 仅使用真实训练面孔对MagFace进行微调；(iii) 使用真实和合成生成的训练面孔组合对MagFace进行微调。利用StyleGAN2 ADA生成的合成数据仅包含在训练身份中，并且在生成后应用了一个过滤步骤以减轻身份泄漏并移除受影响的样本。实验结果显示，在6到36个月的注册验证间隔期间，合成增强型微调相对于预训练基线和仅使用真实面孔进行微调的情况显著降低了错误率。这些发现为提高儿童面部识别中身份持久性的合成数据增强方法的风险评估提供了依据。
URL
https://arxiv.org/abs/2601.01689
PDF
-
Evaluating Deep Learning-Based Face Recognition for Infants and Toddlers: Impact of Age Across Developmental Stages
2026-01-04 22:30:28Afzal Hossain, Mst Rumana Sumi, Stephanie SchuckersAbstract
Face recognition for infants and toddlers presents unique challenges due to rapid facial morphology changes, high inter-class similarity, and limited dataset availability. This study evaluates the performance of four deep learning-based face recognition models FaceNet, ArcFace, MagFace, and CosFace on a newly developed longitudinal dataset collected over a 24 month period in seven sessions involving children aged 0 to 3 years. Our analysis examines recognition accuracy across developmental stages, showing that the True Accept Rate (TAR) is only 30.7% at 0.1% False Accept Rate (FAR) for infants aged 0 to 6 months, due to unstable facial features. Performance improves significantly in older children, reaching 64.7% TAR at 0.1% FAR in the 2.5 to 3 year age group. We also evaluate verification performance over different time intervals, revealing that shorter time gaps result in higher accuracy due to reduced embedding drift. To mitigate this drift, we apply a Domain Adversarial Neural Network (DANN) approach that improves TAR by over 12%, yielding features that are more temporally stable and generalizable. These findings are critical for building biometric systems that function reliably over time in smart city applications such as public healthcare, child safety, and digital identity services. The challenges observed in early age groups highlight the importance of future research on privacy preserving biometric authentication systems that can address temporal variability, particularly in secure and regulated urban environments where child verification is essential.
Abstract (translated)
婴儿和幼儿的面部识别面临着独特的挑战，因为他们的面部形态快速变化、类别间的相似度高以及可用数据量有限。本研究评估了四种基于深度学习的面部识别模型（FaceNet、ArcFace、MagFace 和 CosFace）在一项新开发的纵向数据集上的性能表现。该数据集是在七年时间里分七个阶段收集而成，涉及年龄从0到3岁的儿童。 我们的分析考察了不同发育阶段下的识别准确性，结果显示，在0.1%假阳性率（FAR）条件下，对于0至6个月大的婴儿，真实接受率（TAR）仅为30.7%，这是由于面部特征不稳定所致。随着儿童年龄的增长，性能显著提高，在2.5到3岁的年龄段中，TAR在同样的错误接受率下达到了64.7%。 我们还评估了不同时间间隔下的验证性能，发现较短的时间差会导致更高的准确性，因为这减少了嵌入漂移（即随着时间推移面部特征表示的变化）的影响。为了缓解这种漂移，我们应用了一种域对抗神经网络（DANN）方法，该方法将TAR提高了超过12%，生成了更具有时间稳定性和泛化能力的特性。 这些发现对于构建在智慧城市应用程序中可靠运行的生物识别系统至关重要，例如公共健康、儿童安全和数字身份服务。早期年龄组观察到的挑战突显了未来研究的重要性，特别是关于隐私保护型生物特征认证系统的研究，这类系统能够应对时间变化性问题，在需要严格监管的城市环境中尤为关键，尤其是在儿童验证至关重要的场合下。
URL
https://arxiv.org/abs/2601.01680
PDF
-
Multi-Attribute guided Thermal Face Image Translation based on Latent Diffusion Model
2025-12-24 07:55:54Mingshu Cai, Osamu Yoshie, Yuya IeiriAbstract
Modern surveillance systems increasingly rely on multi-wavelength sensors and deep neural networks to recognize faces in infrared images captured at night. However, most facial recognition models are trained on visible light datasets, leading to substantial performance degradation on infrared inputs due to significant domain shifts. Early feature-based methods for infrared face recognition proved ineffective, prompting researchers to adopt generative approaches that convert infrared images into visible light images for improved recognition. This paradigm, known as Heterogeneous Face Recognition (HFR), faces challenges such as model and modality discrepancies, leading to distortion and feature loss in generated images. To address these limitations, this paper introduces a novel latent diffusion-based model designed to generate high-quality visible face images from thermal inputs while preserving critical identity features. A multi-attribute classifier is incorporated to extract key facial attributes from visible images, mitigating feature loss during infrared-to-visible image restoration. Additionally, we propose the Self-attn Mamba module, which enhances global modeling of cross-modal features and significantly improves inference speed. Experimental results on two benchmark datasets demonstrate the superiority of our approach, achieving state-of-the-art performance in both image quality and identity preservation.
Abstract (translated)
现代监控系统越来越多地依赖多波段传感器和深度神经网络来识别夜间捕捉的红外图像中的面部。然而，大多数面部识别模型是在可见光数据集上训练的，因此在处理红外输入时性能显著下降，这是由于领域偏移（domain shift）造成的。早期基于特征的方法对于红外人脸识别效果不佳，促使研究人员采用生成式方法将红外图像转换为可见光图像以提高识别率。这种范式被称为异质面部识别（Heterogeneous Face Recognition, HFR），它面临着模型和模式差异带来的挑战，导致在生成的图像中出现失真和特征丢失的问题。为了克服这些限制，本文介绍了一种基于潜在扩散的新模型，旨在从热输入生成高质量的可见人脸图像，并保留关键的身份特征。此外，我们还引入了一个多属性分类器来提取来自可见光图像的关键面部特征，从而在红外转可见光图像恢复过程中减少特征丢失。另外，我们提出了Self-attn Mamba模块，它增强了跨模态特征的整体建模能力，并显著提高了推理速度。在两个基准数据集上的实验结果证明了我们方法的优越性，在图像质量和身份保留方面均达到了当前最先进的性能水平。
URL
https://arxiv.org/abs/2512.21032
PDF
-
CLIP-FTI: Fine-Grained Face Template Inversion via CLIP-Driven Attribute Conditioning
2025-12-17 13:26:34Longchen Dai, Zixuan Shen, Zhiheng Zhou, Peipeng Yu, Zhihua XiaAbstract
Face recognition systems store face templates for efficient matching. Once leaked, these templates pose a threat: inverting them can yield photorealistic surrogates that compromise privacy and enable impersonation. Although existing research has achieved relatively realistic face template inversion, the reconstructed facial images exhibit over-smoothed facial-part attributes (eyes, nose, mouth) and limited transferability. To address this problem, we present CLIP-FTI, a CLIP-driven fine-grained attribute conditioning framework for face template inversion. Our core idea is to use the CLIP model to obtain the semantic embeddings of facial features, in order to realize the reconstruction of specific facial feature attributes. Specifically, facial feature attribute embeddings extracted from CLIP are fused with the leaked template via a cross-modal feature interaction network and projected into the intermediate latent space of a pretrained StyleGAN. The StyleGAN generator then synthesizes face images with the same identity as the templates but with more fine-grained facial feature attributes. Experiments across multiple face recognition backbones and datasets show that our reconstructions (i) achieve higher identification accuracy and attribute similarity, (ii) recover sharper component-level attribute semantics, and (iii) improve cross-model attack transferability compared to prior reconstruction attacks. To the best of our knowledge, ours is the first method to use additional information besides the face template attack to realize face template inversion and obtains SOTA results.
Abstract (translated)
面部识别系统存储人脸模板，以实现高效的匹配。一旦这些模板被泄露，它们会构成威胁：通过逆向工程可以生成逼真的面部替身，从而侵犯隐私并可能导致身份冒用。尽管现有的研究已经实现了相对逼真的面部模板逆向过程，但重建的面部图像展示出过度平滑的眼部、鼻子和嘴巴等面部特征属性，并且具有有限的跨模型迁移能力。 为了应对这个问题，我们提出了CLIP-FTI框架，这是一个由CLIP驱动的细粒度属性条件化框架，用于实现面部模板逆向过程。我们的核心理念是利用CLIP模型获取面部特征的语义嵌入，从而能够重构特定的面部特征属性。具体而言，从CLIP中提取出的面部特征属性嵌入通过跨模态特征交互网络与泄露的人脸模板融合，并投影到预先训练过的StyleGAN的中间潜在空间内。随后，StyleGAN生成器会合成具有与原始模板相同身份但拥有更精细的面部特征属性的脸部图像。 在多个面部识别骨干架构和数据集上的实验显示，我们的重建方法实现了以下改进：（i）更高的识别准确性和属性相似度；（ii）恢复了更为清晰、细节层次的语义信息；以及（iii）跨模型攻击迁移性的提升。据我们所知，这是首个使用除人脸模板之外额外信息来实现面部模板逆向过程的方法，并且在现有技术中取得了最佳结果（SOTA）。
URL
https://arxiv.org/abs/2512.15433
PDF
-
Face Identity Unlearning for Retrieval via Embedding Dispersion
2025-12-15 13:35:28Mikhail ZakharovAbstract
Face recognition systems rely on learning highly discriminative and compact identity clusters to enable accurate retrieval. However, as with other surveillance-oriented technologies, such systems raise serious privacy concerns due to their potential for unauthorized identity tracking. While several works have explored machine unlearning as a means of privacy protection, their applicability to face retrieval - especially for modern embedding-based recognition models - remains largely unexplored. In this work, we study the problem of face identity unlearning for retrieval systems and present its inherent challenges. The goal is to make selected identities unretrievable by dispersing their embeddings on the hypersphere and preventing the formation of compact identity clusters that enable re-identification in the gallery. The primary challenge is to achieve this forgetting effect while preserving the discriminative structure of the embedding space and the retrieval performance of the model for the remaining identities. To address this, we evaluate several existing approximate class unlearning methods (e.g., Random Labeling, Gradient Ascent, Boundary Unlearning, and other recent approaches) in the context of face retrieval and propose a simple yet effective dispersion-based unlearning approach. Extensive experiments on standard benchmarks (VGGFace2, CelebA) demonstrate that our method achieves superior forgetting behavior while preserving retrieval utility.
Abstract (translated)
面部识别系统依赖于学习高度判别性和紧凑的身份聚类，以实现准确的检索。然而，与其他监控技术一样，这类系统由于其潜在的未经授权的身份追踪能力而引发了严重的隐私问题。尽管已有几项研究探索了机器遗忘作为隐私保护手段的应用，但它们在面部检索（尤其是针对现代嵌入式识别模型）中的适用性仍然鲜有探讨。 在这项工作中，我们研究了面向检索系统的面部身份遗忘问题，并展示了其固有的挑战。我们的目标是通过分散选定身份的嵌入以使其无法被检索，在超球面上防止紧凑的身份聚类形成，从而阻止重新识别。主要挑战在于在保留嵌入空间的判别性结构和模型对剩余身份的检索性能的同时实现这种忘记效果。 为解决这一问题，我们评估了几种现有的近似类别遗忘方法（如随机标签法、梯度上升法、边界遗忘法及其他最近的方法）在面部检索中的应用，并提出了一种简单而有效的基于分散的遗忘策略。我们在标准基准测试集（VGGFace2和CelebA）上的广泛实验表明，我们的方法能够实现更优的忘记行为同时保持检索效用。
URL
https://arxiv.org/abs/2512.13317
PDF
-
Two-Step Data Augmentation for Masked Face Detection and Recognition: Turning Fake Masks to Real
2025-12-13 15:20:08Yan Yang, George Bebis, Mircea NicolescuAbstract
Data scarcity and distribution shift pose major challenges for masked face detection and recognition. We propose a two-step generative data augmentation framework that combines rule-based mask warping with unpaired image-to-image translation using GANs, enabling the generation of realistic masked-face samples beyond purely synthetic transformations. Compared to rule-based warping alone, the proposed approach yields consistent qualitative improvements and complements existing GAN-based masked face generation methods such as IAMGAN. We introduce a non-mask preservation loss and stochastic noise injection to stabilize training and enhance sample diversity. Experimental observations highlight the effectiveness of the proposed components and suggest directions for future improvements in data-centric augmentation for face recognition tasks.
Abstract (translated)
数据稀缺和分布变化给戴口罩人脸检测与识别带来了重大挑战。为此，我们提出了一种两步生成式数据增强框架，该框架结合了基于规则的口罩变形技术与未配对图像到图像转换（使用GAN）的方法，能够生成超越单纯合成变换的真实戴口罩人脸样本。相较于仅采用基于规则的变形方法，所提出的方案在定性上持续改进，并补充现有的如IAMGAN等基于GAN的戴口罩人脸生成方法。我们引入了非面具保持损失和随机噪声注入来稳定训练并增强样本多样性。实验观察结果强调了所提出组件的有效性，并为未来以数据为中心的数据增强技术在人脸识别任务中的改进方向提供了建议。
URL
https://arxiv.org/abs/2512.15774
PDF
-
Detection of Digital Facial Retouching utilizing Face Beauty Information
2025-12-09 09:23:33Philipp Srock, Juan E. Tapia, Christoph BuschAbstract
Facial retouching to beautify images is widely spread in social media, advertisements, and it is even applied in professional photo studios to let individuals appear younger, remove wrinkles and skin impurities. Generally speaking, this is done to enhance beauty. This is not a problem itself, but when retouched images are used as biometric samples and enrolled in a biometric system, it is one. Since previous work has proven facial retouching to be a challenge for face recognition systems,the detection of facial retouching becomes increasingly necessary. This work proposes to study and analyze changes in beauty assessment algorithms of retouched images, assesses different feature extraction methods based on artificial intelligence in order to improve retouching detection, and evaluates whether face beauty can be exploited to enhance the detection rate. In a scenario where the attacking retouching algorithm is unknown, this work achieved 1.1% D-EER on single image detection.
Abstract (translated)
面部修图在社交媒体、广告中非常普遍，甚至专业摄影棚也使用这种方法让个人看起来更年轻、去除皱纹和皮肤瑕疵。通常来说，这种做法是为了提升美感，并无大碍；然而，当经过修饰的图像被用作生物识别样本并录入到生物识别系统时，问题就出现了。之前的研究已经证明面部修图对人脸识别系统构成了挑战，因此检测面部修图变得越来越必要。 本研究旨在探讨和分析修图后图像在美容评估算法中的变化，评估基于人工智能的不同特征提取方法以提高修图检测的准确性，并检验脸部美感是否可以被利用来提升检测率。在攻击性修图算法未知的情境下，该研究仅凭单张图片检测就能达到1.1%的虚警误报率（D-EER）。
URL
https://arxiv.org/abs/2512.08397
PDF
-
Spoofing-aware Prompt Learning for Unified Physical-Digital Facial Attack Detection
2025-12-06 09:34:39Jiabao Guo, Yadian Wang, Hui Ma, Yuhao Fu, Ju Jia, Hui Liu, Shengeng Tang, Lechao Cheng, Yunfeng Diao, Ajian LiuAbstract
Real-world face recognition systems are vulnerable to both physical presentation attacks (PAs) and digital forgery attacks (DFs). We aim to achieve comprehensive protection of biometric data by implementing a unified physical-digital defense framework with advanced detection. Existing approaches primarily employ CLIP with regularization constraints to enhance model generalization across both tasks. However, these methods suffer from conflicting optimization directions between physical and digital attack detection under same category prompt spaces. To overcome this limitation, we propose a Spoofing-aware Prompt Learning for Unified Attack Detection (SPL-UAD) framework, which decouples optimization branches for physical and digital attacks in the prompt space. Specifically, we construct a learnable parallel prompt branch enhanced with adaptive Spoofing Context Prompt Generation, enabling independent control of optimization for each attack type. Furthermore, we design a Cues-awareness Augmentation that leverages the dual-prompt mechanism to generate challenging sample mining tasks on data, significantly enhancing the model's robustness against unseen attack types. Extensive experiments on the large-scale UniAttackDataPlus dataset demonstrate that the proposed method achieves significant performance improvements in unified attack detection tasks.
Abstract (translated)
现实世界中的面部识别系统容易受到物理展示攻击（PA）和数字伪造攻击（DF）的威胁。我们的目标是通过实施一个具有高级检测功能的统一物理-数字防御框架来实现生物特征数据的全面保护。现有的方法主要采用带有正则化约束条件的CLIP，以增强模型在两类任务中的泛化能力。然而，这些方法在同类别提示空间中存在针对物理和数字攻击检测优化方向冲突的问题。 为了解决这一局限性，我们提出了一个名为“基于伪造意识提示学习的一体化攻击检测框架”（SPL-UAD），该框架在提示空间中解耦了对物理和数字攻击的优化分支。具体而言，我们构建了一个可以通过自适应伪造上下文提示生成来增强的可学习并行提示分支，从而能够独立控制每种攻击类型的优化过程。 此外，我们设计了一种基于线索感知的数据增强机制，利用双提示机制在数据上生成挑战性的样本挖掘任务，显著提高了模型对未见过的攻击类型时的鲁棒性。在大规模UniAttackDataPlus数据集上的广泛实验表明，所提出的方法在统一攻击检测任务中取得了显著的性能改进。
URL
https://arxiv.org/abs/2512.06363
PDF
-
A Comparative Study on Synthetic Facial Data Generation Techniques for Face Recognition
2025-12-05 18:11:29Pedro Vidal, Bernardo Biesseck, Luiz E. L. Coelho, Roger Granada, David MenottiAbstract
Facial recognition has become a widely used method for authentication and identification, with applications for secure access and locating missing persons. Its success is largely attributed to deep learning, which leverages large datasets and effective loss functions to learn discriminative features. Despite these advances, facial recognition still faces challenges in explainability, demographic bias, privacy, and robustness to aging, pose variations, lighting changes, occlusions, and facial expressions. Privacy regulations have also led to the degradation of several datasets, raising legal, ethical, and privacy concerns. Synthetic facial data generation has been proposed as a promising solution. It mitigates privacy issues, enables experimentation with controlled facial attributes, alleviates demographic bias, and provides supplementary data to improve models trained on real data. This study compares the effectiveness of synthetic facial datasets generated using different techniques in facial recognition tasks. We evaluate accuracy, rank-1, rank-5, and the true positive rate at a false positive rate of 0.01% on eight leading datasets, offering a comparative analysis not extensively explored in the literature. Results demonstrate the ability of synthetic data to capture realistic variations while emphasizing the need for further research to close the performance gap with real data. Techniques such as diffusion models, GANs, and 3D models show substantial progress; however, challenges remain.
Abstract (translated)
面部识别已成为认证和身份识别中广泛应用的方法，应用于安全访问和个人寻找等场景。其成功主要归功于深度学习技术，该技术利用大规模数据集和有效的损失函数来学习区分性特征。尽管取得了这些进展，面部识别仍然面临着解释性、人口统计学偏差、隐私以及应对年龄变化、姿势变换、光照改变、遮挡及表情差异等方面的挑战。随着隐私法规的出台，许多数据集也因此遭到破坏或无法使用，引发了法律、伦理和隐私方面的担忧。合成面部数据生成被提出作为解决这些问题的一个有前景的方法。它可以减轻隐私问题，使人们能够在控制的人脸属性条件下进行实验，缓解人口统计学偏差，并提供补充数据以改进基于真实数据训练的模型。本研究比较了采用不同技术生成的合成面部数据集在面部识别任务中的有效性。我们在八个领先的数据集中评估了准确率、第一匹配率（rank-1）、第五匹配率（rank-5）以及在0.01%错误接受率下的真阳性率，提供了文献中未被广泛探讨的一个对比分析。结果表明，合成数据能够捕捉到现实变化的特征，同时也强调了需要进一步研究以缩小与真实数据性能之间的差距。扩散模型、GANs和3D模型等技术已经取得了显著的进步；然而，依然存在挑战有待克服。
URL
https://arxiv.org/abs/2512.05928
PDF
-
Mitigating Bias with Words: Inducing Demographic Ambiguity in Face Recognition Templates by Text Encoding
2025-12-05 10:41:58Tahar Chettaoui, Naser Damer, Fadi BoutrosAbstract
Face recognition (FR) systems are often prone to demographic biases, partially due to the entanglement of demographic-specific information with identity-relevant features in facial embeddings. This bias is extremely critical in large multicultural cities, especially where biometrics play a major role in smart city infrastructure. The entanglement can cause demographic attributes to overshadow identity cues in the embedding space, resulting in disparities in verification performance across different demographic groups. To address this issue, we propose a novel strategy, Unified Text-Image Embedding (UTIE), which aims to induce demographic ambiguity in face embeddings by enriching them with information related to other demographic groups. This encourages face embeddings to emphasize identity-relevant features and thus promotes fairer verification performance across groups. UTIE leverages the zero-shot capabilities and cross-modal semantic alignment of Vision-Language Models (VLMs). Given that VLMs are naturally trained to align visual and textual representations, we enrich the facial embeddings of each demographic group with text-derived demographic features extracted from other demographic groups. This encourages a more neutral representation in terms of demographic attributes. We evaluate UTIE using three VLMs, CLIP, OpenCLIP, and SigLIP, on two widely used benchmarks, RFW and BFW, designed to assess bias in FR. Experimental results show that UTIE consistently reduces bias metrics while maintaining, or even improving in several cases, the face verification accuracy.
Abstract (translated)
人脸识别（FR）系统常常存在人口统计学偏差，部分原因是面部嵌入中包含了特定于某一人群的信息与身份相关特征的纠缠。这种偏见在大型多元文化城市尤其严重，尤其是在生物识别技术对智慧城市基础设施发挥重要作用的情况下。这种纠缠可能导致人口统计属性在嵌入空间中压倒了身份线索，从而导致不同人口群体之间的验证性能差异。为解决这一问题，我们提出了一种新颖策略——统一文本图像嵌入（UTIE），旨在通过将与其它人口群体相关的特征信息加入面部嵌入来诱导出人口统计学的模糊性。这鼓励面部嵌入强调身份相关特征，从而促进不同群体间更公平的验证性能。 UTIE利用了视觉语言模型（VLM）的零样本能力和跨模态语义对齐能力。由于VLM自然地训练用于调整视觉和文本表示的一致性，我们用从其它人口统计学群组提取出的文字衍生的人口特征来丰富每个群体的面部嵌入。这鼓励了一个更中立的人口属性表示。 我们在CLIP、OpenCLIP 和 SigLIP 三种 VLM 上使用 RFW 和 BFW 这两个广泛使用的基准进行UTIE评估，后者设计用于测试 FR 中的偏见问题。实验结果显示 UTIE 在降低偏见指标的同时保持甚至在某些情况下提高了面部验证准确率。
URL
https://arxiv.org/abs/2512.08981
PDF
