Face_Recognition
-
Superpixel Attack: Enhancing Black-box Adversarial Attack with Image-driven Division Areas
2025-11-29 05:28:52Issa Oe, Keiichiro Yamamura, Hiroki Ishikura, Ryo Hamahira, Katsuki FujisawaAbstract
Deep learning models are used in safety-critical tasks such as automated driving and face recognition. However, small perturbations in the model input can significantly change the predictions. Adversarial attacks are used to identify small perturbations that can lead to misclassifications. More powerful black-box adversarial attacks are required to develop more effective defenses. A promising approach to black-box adversarial attacks is to repeat the process of extracting a specific image area and changing the perturbations added to it. Existing attacks adopt simple rectangles as the areas where perturbations are changed in a single iteration. We propose applying superpixels instead, which achieve a good balance between color variance and compactness. We also propose a new search method, versatile search, and a novel attack method, Superpixel Attack, which applies superpixels and performs versatile search. Superpixel Attack improves attack success rates by an average of 2.10% compared with existing attacks. Most models used in this study are robust against adversarial attacks, and this improvement is significant for black-box adversarial attacks. The code is avilable at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2512.02062
PDF
-
GFT-GCN: Privacy-Preserving 3D Face Mesh Recognition with Spectral Diffusion
2025-11-25 06:07:26Hichem Felouat, Hanrui Wang, Isao EchizenAbstract
3D face recognition offers a robust biometric solution by capturing facial geometry, providing resilience to variations in illumination, pose changes, and presentation attacks. Its strong spoof resistance makes it suitable for high-security applications, but protecting stored biometric templates remains critical. We present GFT-GCN, a privacy-preserving 3D face recognition framework that combines spectral graph learning with diffusion-based template protection. Our approach integrates the Graph Fourier Transform (GFT) and Graph Convolutional Networks (GCN) to extract compact, discriminative spectral features from 3D face meshes. To secure these features, we introduce a spectral diffusion mechanism that produces irreversible, renewable, and unlinkable templates. A lightweight client-server architecture ensures that raw biometric data never leaves the client device. Experiments on the BU-3DFE and FaceScape datasets demonstrate high recognition accuracy and strong resistance to reconstruction attacks. Results show that GFT-GCN effectively balances privacy and performance, offering a practical solution for secure 3D face authentication.
Abstract (translated)
URL
https://arxiv.org/abs/2511.19958
PDF
-
Face, Whole-Person, and Object Classification in a Unified Space Via The Interleaved Multi-Domain Identity Curriculum
2025-11-25 02:23:10Thomas M Metz, Matthew Q Hill, Alice J O'TooleAbstract
Vision foundation models can perform generalized object classification in zero-shot mode, and face/person recognition when they are fine-tuned. However, fine-tuned models suffer from catastrophic forgetting. We create models that perform four tasks (object recognition, face recognition from high- and low-quality images, and person recognition from whole-body images) in a single embedding space -- without incurring substantial catastrophic forgetting. To accomplish this, we introduce two variants of the Interleaved Multi-Domain Identity Curriculum (IMIC): a gradient-coupled, interleaving training schedule that fine-tunes a foundation backbone simultaneously on all four tasks. The IMIC method proved effective with three foundation model bases: DINOv3, CLIP, and EVA-02. Two of these (EVA-02 and CLIP) performed comparably with domain experts on all four tasks concurrently and were more accurate than humans at multi-tasking across face, body, and object datasets. Further, we demonstrate that our approach does not substantially harm out-of-distribution generalization, thus maintaining a key property of foundation models. Analysis of the most accurate model variants (EVA-02 + IMIC A and B) showed linearly separable representations of the four tasks in the unified embedding space, but with substantial sharing of features across tasks. Fewer than 100 PCs calculated from any one task could perform all other tasks with nearly zero performance degradation.
Abstract (translated)
URL
https://arxiv.org/abs/2511.19846
PDF
-
Quality-Controlled Multimodal Emotion Recognition in Conversations with Identity-Based Transfer Learning and MAMBA Fusion
2025-11-18 23:24:27Zanxu Wang, Homayoon BeigiAbstract
This paper addresses data quality issues in multimodal emotion recognition in conversation (MERC) through systematic quality control and multi-stage transfer learning. We implement a quality control pipeline for MELD and IEMOCAP datasets that validates speaker identity, audio-text alignment, and face detection. We leverage transfer learning from speaker and face recognition, assuming that identity-discriminative embeddings capture not only stable acoustic and Facial traits but also person-specific patterns of emotional expression. We employ RecoMadeEasy(R) engines for extracting 512-dimensional speaker and face embeddings, fine-tune MPNet-v2 for emotion-aware text representations, and adapt these features through emotion-specific MLPs trained on unimodal datasets. MAMBA-based trimodal fusion achieves 64.8% accuracy on MELD and 74.3% on IEMOCAP. These results show that combining identity-based audio and visual embeddings with emotion-tuned text representations on a quality-controlled subset of data yields consistent competitive performance for multimodal emotion recognition in conversation and provides a basis for further improvement on challenging, low-frequency emotion classes.
Abstract (translated)
URL
https://arxiv.org/abs/2511.14969
PDF
-
LoRA-Enhanced Vision Transformer for Single Image based Morphing Attack Detection via Knowledge Distillation from EfficientNet
2025-11-16 13:58:11Ria Shekhawat, Sushrut Patwardhan, Raghavendra Ramachandra, Praveen Kumar Chandaliya, Kishor P. UplaAbstract
Face Recognition Systems (FRS) are critical for security but remain vulnerable to morphing attacks, where synthetic images blend biometric features from multiple individuals. We propose a novel Single-Image Morphing Attack Detection (S-MAD) approach using a teacher-student framework, where a CNN-based teacher model refines a ViT-based student model. To improve efficiency, we integrate Low-Rank Adaptation (LoRA) for fine-tuning, reducing computational costs while maintaining high detection accuracy. Extensive experiments are conducted on a morphing dataset built from three publicly available face datasets, incorporating ten different morphing generation algorithms to assess robustness. The proposed method is benchmarked against six state-of-the-art S-MAD techniques, demonstrating superior detection performance and computational efficiency.
Abstract (translated)
URL
https://arxiv.org/abs/2511.12602
PDF
-
Batch Transformer Architecture: Case of Synthetic Image Generation for Emotion Expression Facial Recognition
2025-11-13 19:45:41Stanislav SelitskiyAbstract
A novel Transformer variation architecture is proposed in the implicit sparse style. Unlike "traditional" Transformers, instead of attention to sequential or batch entities in their entirety of whole dimensionality, in the proposed Batch Transformers, attention to the "important" dimensions (primary components) is implemented. In such a way, the "important" dimensions or feature selection allows for a significant reduction of the bottleneck size in the encoder-decoder ANN architectures. The proposed architecture is tested on the synthetic image generation for the face recognition task in the case of the makeup and occlusion data set, allowing for increased variability of the limited original data set.
Abstract (translated)
URL
https://arxiv.org/abs/2511.11754
PDF
-
Introducing Nylon Face Mask Attacks: A Dataset for Evaluating Generalised Face Presentation Attack Detection
2025-11-11 11:13:12Manasa, Sushrut Patwardhan, Narayan Vetrekar, Pavan Kumar, R. S. Gad, Raghavendra RamachandraAbstract
Face recognition systems are increasingly deployed across a wide range of applications, including smartphone authentication, access control, and border security. However, these systems remain vulnerable to presentation attacks (PAs), which can significantly compromise their reliability. In this work, we introduce a new dataset focused on a novel and realistic presentation attack instrument called Nylon Face Masks (NFMs), designed to simulate advanced 3D spoofing scenarios. NFMs are particularly concerning due to their elastic structure and photorealistic appearance, which enable them to closely mimic the victim's facial geometry when worn by an attacker. To reflect real-world smartphone-based usage conditions, we collected the dataset using an iPhone 11 Pro, capturing 3,760 bona fide samples from 100 subjects and 51,281 NFM attack samples across four distinct presentation scenarios involving both humans and mannequins. We benchmark the dataset using five state-of-the-art PAD methods to evaluate their robustness under unseen attack conditions. The results demonstrate significant performance variability across methods, highlighting the challenges posed by NFMs and underscoring the importance of developing PAD techniques that generalise effectively to emerging spoofing threats.
Abstract (translated)
URL
https://arxiv.org/abs/2511.08114
PDF
-
StableMorph: High-Quality Face Morph Generation with Stable Diffusion
2025-11-11 10:44:41Wassim Kabbani, Kiran Raja, Raghavendra Ramachandra, Christoph BuschAbstract
Face morphing attacks threaten the integrity of biometric identity systems by enabling multiple individuals to share a single identity. To develop and evaluate effective morphing attack detection (MAD) systems, we need access to high-quality, realistic morphed images that reflect the challenges posed in real-world scenarios. However, existing morph generation methods often produce images that are blurry, riddled with artifacts, or poorly constructed making them easy to detect and not representative of the most dangerous attacks. In this work, we introduce StableMorph, a novel approach that generates highly realistic, artifact-free morphed face images using modern diffusion-based image synthesis. Unlike prior methods, StableMorph produces full-head images with sharp details, avoids common visual flaws, and offers unmatched control over visual attributes. Through extensive evaluation, we show that StableMorph images not only rival or exceed the quality of genuine face images but also maintain a strong ability to fool face recognition systems posing a greater challenge to existing MAD solutions and setting a new standard for morph quality in research and operational testing. StableMorph improves the evaluation of biometric security by creating more realistic and effective attacks and supports the development of more robust detection systems.
Abstract (translated)
URL
https://arxiv.org/abs/2511.08090
PDF
-
Hybrid second-order gradient histogram based global low-rank sparse regression for robust face recognition
2025-11-08 07:31:38Hongxia Li, Ying Ji, Yongxin Dong, Yuehua FengAbstract
Low-rank sparse regression models have been widely applied in the field of face recognition. To further address the challenges caused by complex occlusions and illumination variations, this paper proposes a Hybrid Second-Order Gradient Histogram based Global Low-Rank Sparse Regression (H2H-GLRSR) model. Specifically, a novel feature descriptor called the Hybrid Second-Order Gradient Histogram (H2H) is first designed to more effectively characterize the local structural features of facial images. Then, this descriptor is integrated with the Sparse Regularized Nuclear Norm based Matrix Regression (SR$\_$NMR). Moreover, a global low-rank constraint is imposed on the residual matrix, enabling the model to better capture the global correlations inherent in structured noise. Experimental results demonstrate that the proposed method significantly outperforms existing regression-based classification approaches under challenging scenarios involving occlusions, illumination changes, and unconstrained environments.
Abstract (translated)
URL
https://arxiv.org/abs/2511.05893
PDF
-
Photo Dating by Facial Age Aggregation
2025-11-07 18:08:04Jakub Paplham, Vojtech FrancAbstract
We introduce a novel method for Photo Dating which estimates the year a photograph was taken by leveraging information from the faces of people present in the image. To facilitate this research, we publicly release CSFD-1.6M, a new dataset containing over 1.6 million annotated faces, primarily from movie stills, with identity and birth year annotations. Uniquely, our dataset provides annotations for multiple individuals within a single image, enabling the study of multi-face information aggregation. We propose a probabilistic framework that formally combines visual evidence from modern face recognition and age estimation models, and career-based temporal priors to infer the photo capture year. Our experiments demonstrate that aggregating evidence from multiple faces consistently improves the performance and the approach significantly outperforms strong, scene-based baselines, particularly for images containing several identifiable individuals.
Abstract (translated)
URL
https://arxiv.org/abs/2511.05464
PDF
-
Elements of Active Continuous Learning and Uncertainty Self-Awareness: a Narrow Implementation for Face and Facial Expression Recognition
2025-11-04 17:01:53Stanislav SelitskiyAbstract
Reflection on one's thought process and making corrections to it if there exists dissatisfaction in its performance is, perhaps, one of the essential traits of intelligence. However, such high-level abstract concepts mandatory for Artificial General Intelligence can be modelled even at the low level of narrow Machine Learning algorithms. Here, we present the self-awareness mechanism emulation in the form of a supervising artificial neural network (ANN) observing patterns in activations of another underlying ANN in a search for indications of the high uncertainty of the underlying ANN and, therefore, the trustworthiness of its predictions. The underlying ANN is a convolutional neural network (CNN) ensemble employed for face recognition and facial expression tasks. The self-awareness ANN has a memory region where its past performance information is stored, and its learnable parameters are adjusted during the training to optimize the performance. The trustworthiness verdict triggers the active learning mode, giving elements of agency to the machine learning algorithm that asks for human help in high uncertainty and confusion conditions.
Abstract (translated)
URL
https://arxiv.org/abs/2511.05574
PDF
-
Robust Face Liveness Detection for Biometric Authentication using Single Image
2025-11-04 15:13:03Poulami Raha, Yeongnam ChaeAbstract
Biometric technologies are widely adopted in security, legal, and financial systems. Face recognition can authenticate a person based on the unique facial features such as shape and texture. However, recent works have demonstrated the vulnerability of Face Recognition Systems (FRS) towards presentation attacks. Using spoofing (aka.,presentation attacks), a malicious actor can get illegitimate access to secure systems. This paper proposes a novel light-weight CNN framework to identify print/display, video and wrap attacks. The proposed robust architecture provides seamless liveness detection ensuring faster biometric authentication (1-2 seconds on CPU). Further, this also presents a newly created 2D spoof attack dataset consisting of more than 500 videos collected from 60 subjects. To validate the effectiveness of this architecture, we provide a demonstration video depicting print/display, video and wrap attack detection approaches. The demo can be viewed in the following link: this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2511.02645
PDF
-
PSTF-AttControl: Per-Subject-Tuning-Free Personalized Image Generation with Controllable Face Attributes
2025-10-29 01:42:23Xiang liu, Zhaoxiang Liu, Huan Hu, Zipeng Wang, Ping Chen, Zezhou Chen, Kai Wang, Shiguo LianAbstract
Recent advancements in personalized image generation have significantly improved facial identity preservation, particularly in fields such as entertainment and social media. However, existing methods still struggle to achieve precise control over facial attributes in a per-subject-tuning-free (PSTF) way. Tuning-based techniques like PreciseControl have shown promise by providing fine-grained control over facial features, but they often require extensive technical expertise and additional training data, limiting their accessibility. In contrast, PSTF approaches simplify the process by enabling image generation from a single facial input, but they lack precise control over facial attributes. In this paper, we introduce a novel, PSTF method that enables both precise control over facial attributes and high-fidelity preservation of facial identity. Our approach utilizes a face recognition model to extract facial identity features, which are then mapped into the $W^+$ latent space of StyleGAN2 using the e4e encoder. We further enhance the model with a Triplet-Decoupled Cross-Attention module, which integrates facial identity, attribute features, and text embeddings into the UNet architecture, ensuring clean separation of identity and attribute information. Trained on the FFHQ dataset, our method allows for the generation of personalized images with fine-grained control over facial attributes, while without requiring additional fine-tuning or training data for individual identities. We demonstrate that our approach successfully balances personalization with precise facial attribute control, offering a more efficient and user-friendly solution for high-quality, adaptable facial image synthesis. The code is publicly available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2510.25084
PDF
-
Switchable Token-Specific Codebook Quantization For Face Image Compression
2025-10-27 02:56:17Yongbo Wang, Haonan Wang, Guodong Mu, Ruixin Zhang, Jiaqi Chen, Jingyun Zhang, Jun Wang, Yuan Xie, Zhizhong Zhang, Shouhong DingAbstract
With the ever-increasing volume of visual data, the efficient and lossless transmission, along with its subsequent interpretation and understanding, has become a critical bottleneck in modern information systems. The emerged codebook-based solution utilize a globally shared codebook to quantize and dequantize each token, controlling the bpp by adjusting the number of tokens or the codebook size. However, for facial images, which are rich in attributes, such global codebook strategies overlook both the category-specific correlations within images and the semantic differences among tokens, resulting in suboptimal performance, especially at low bpp. Motivated by these observations, we propose a Switchable Token-Specific Codebook Quantization for face image compression, which learns distinct codebook groups for different image categories and assigns an independent codebook to each token. By recording the codebook group to which each token belongs with a small number of bits, our method can reduce the loss incurred when decreasing the size of each codebook group. This enables a larger total number of codebooks under a lower overall bpp, thereby enhancing the expressive capability and improving reconstruction performance. Owing to its generalizable design, our method can be integrated into any existing codebook-based representation learning approach and has demonstrated its effectiveness on face recognition datasets, achieving an average accuracy of 93.51% for reconstructed images at 0.05 bpp.
Abstract (translated)
URL
https://arxiv.org/abs/2510.22943
PDF
-
Reliable and Reproducible Demographic Inference for Fairness in Face Analysis
2025-10-23 12:22:02Alexandre Fournier-Montgieux, Herv\'e Le Borgne, Adrian Popescu, Bertrand LuvisonAbstract
Fairness evaluation in face analysis systems (FAS) typically depends on automatic demographic attribute inference (DAI), which itself relies on predefined demographic segmentation. However, the validity of fairness auditing hinges on the reliability of the DAI process. We begin by providing a theoretical motivation for this dependency, showing that improved DAI reliability leads to less biased and lower-variance estimates of FAS fairness. To address this, we propose a fully reproducible DAI pipeline that replaces conventional end-to-end training with a modular transfer learning approach. Our design integrates pretrained face recognition encoders with non-linear classification heads. We audit this pipeline across three dimensions: accuracy, fairness, and a newly introduced notion of robustness, defined via intra-identity consistency. The proposed robustness metric is applicable to any demographic segmentation scheme. We benchmark the pipeline on gender and ethnicity inference across multiple datasets and training setups. Our results show that the proposed method outperforms strong baselines, particularly on ethnicity, which is the more challenging attribute. To promote transparency and reproducibility, we will publicly release the training dataset metadata, full codebase, pretrained models, and evaluation toolkit. This work contributes a reliable foundation for demographic inference in fairness auditing.
Abstract (translated)
面部分析系统（FAS）中的公平性评估通常依赖于自动人口属性推断（DAI），而后者又基于预定义的人口细分。然而，公平审计的有效性取决于DAI过程的可靠性。我们首先提供这种依赖关系的理论动机，并表明改进了DAI的可靠性可以减少对FAS公平性的偏差和方差估计。为了解决这一问题，我们提出了一种完全可重现的DAI流水线，该流水线用模块化的迁移学习方法代替传统的端到端训练。我们的设计将预训练的人脸识别编码器与非线性分类头部相结合。我们在准确率、公平性和一种新引入的鲁棒性的概念上对这一流程进行审核，后者通过同一身份内的一致性来定义。我们提出的鲁棒性度量适用于任何人口细分方案。 在多个数据集和训练设置下，我们对该流水线进行了性别和族裔推断基准测试。实验结果显示，所提出的方法优于强大的基线方法，在更难处理的族裔属性上尤其如此。为了促进透明性和可重现性，我们将公开发布用于训练的数据集元信息、完整代码库、预训练模型以及评估工具包。 这项工作为公平审计中的人口统计推断提供了一个可靠的基石。
URL
https://arxiv.org/abs/2510.20482
PDF
-
Optimizing DINOv2 with Registers for Face Anti-Spoofing
2025-10-20 06:27:02Mika Feng, Pierre Gallin-Martel, Koichi Ito, Takafumi AokiAbstract
Face recognition systems are designed to be robust against variations in head pose, illumination, and image blur during capture. However, malicious actors can exploit these systems by presenting a face photo of a registered user, potentially bypassing the authentication process. Such spoofing attacks must be detected prior to face recognition. In this paper, we propose a DINOv2-based spoofing attack detection method to discern minute differences between live and spoofed face images. Specifically, we employ DINOv2 with registers to extract generalizable features and to suppress perturbations in the attention mechanism, which enables focused attention on essential and minute features. We demonstrate the effectiveness of the proposed method through experiments conducted on the dataset provided by ``The 6th Face Anti-Spoofing Workshop: Unified Physical-Digital Attacks Detection@ICCV2025'' and SiW dataset.
Abstract (translated)
面部识别系统被设计成能够抵抗捕捉过程中头部姿态、光照变化和图像模糊带来的影响。然而，恶意行为者可以通过向系统展示已注册用户的面部照片来利用这些系统，从而可能绕过身份验证过程。这样的伪装攻击必须在进行面部识别之前就被检测到。 本文提出了一种基于DINOv2的伪造攻击检测方法，用于区分真实人脸和伪造人脸图像之间的细微差别。具体来说，我们使用带有登记功能的DINOv2来提取可泛化的特征，并抑制注意力机制中的扰动，从而能够在关键和微小细节上集中注意力。 通过在由“第六届面部防伪研讨会：统一物理-数字攻击检测@ICCV2025”提供的数据集以及SiW数据集上的实验，我们展示了所提出方法的有效性。
URL
https://arxiv.org/abs/2510.17201
PDF
-
Investigating Adversarial Robustness against Preprocessing used in Blackbox Face Recognition
2025-10-20 05:19:35Roland Croft, Brian Du, Darcy Joseph, Sharath KumarAbstract
Face Recognition (FR) models have been shown to be vulnerable to adversarial examples that subtly alter benign facial images, exposing blind spots in these systems, as well as protecting user privacy. End-to-end FR systems first obtain preprocessed faces from diverse facial imagery prior to computing the similarity of the deep feature embeddings. Whilst face preprocessing is a critical component of FR systems, and hence adversarial attacks against them, we observe that this preprocessing is often overlooked in blackbox settings. Our study seeks to investigate the transferability of several out-of-the-box state-of-the-art adversarial attacks against FR when applied against different preprocessing techniques used in a blackbox setting. We observe that the choice of face detection model can degrade the attack success rate by up to 78%, whereas choice of interpolation method during downsampling has relatively minimal impacts. Furthermore, we find that the requirement for facial preprocessing even degrades attack strength in a whitebox setting, due to the unintended interaction of produced noise vectors against face detection models. Based on these findings, we propose a preprocessing-invariant method using input transformations that improves the transferability of the studied attacks by up to 27%. Our findings highlight the importance of preprocessing in FR systems, and the need for its consideration towards improving the adversarial generalisation of facial adversarial examples.
Abstract (translated)
面部识别（FR）模型已被证明容易受到微小改变的良性面部图像攻击，这些攻击揭示了系统的漏洞，并保护了用户隐私。端到端的面部识别系统在计算深度特征嵌入相似性之前，会从各种面部图像中获取预处理过的脸部。虽然面部预处理是面部识别系统的一个关键组成部分，因此也是对抗攻击的一部分，但我们观察到，在黑盒设置下，这一过程往往被忽视了。我们的研究旨在探讨几种现成的最先进的对抗攻击在应用于不同黑盒设置中的预处理技术时的效果转移性。 我们发现，选择不同的面部检测模型可以降低78%的攻击成功率，而降采样期间使用的插值方法的影响相对较小。此外，我们还发现对预处理的需求甚至会削弱白盒环境下的攻击力度，这是由于生成的噪声向量与面部检测模型之间的意外相互作用所致。 基于这些发现，我们提出了一种使用输入变换的方法来提高所研究攻击的效果转移性，这种方法不受预处理过程的影响，并将效果提高了多达27%。我们的研究表明，在面部识别系统中，预处理的重要性不容忽视，并且为了改善面部对抗样本的泛化能力，需要考虑这一因素。
URL
https://arxiv.org/abs/2510.17169
PDF
-
Latent Feature Alignment: Discovering Biased and Interpretable Subpopulations in Face Recognition Models
Abstract
Modern face recognition models achieve high overall accuracy but continue to exhibit systematic biases that disproportionately affect certain subpopulations. Conventional bias evaluation frameworks rely on labeled attributes to form subpopulations, which are expensive to obtain and limited to predefined categories. We introduce Latent Feature Alignment (LFA), an attribute-label-free algorithm that uses latent directions to identify subpopulations. This yields two main benefits over standard clustering: (i) semantically coherent grouping, where faces sharing common attributes are grouped together more reliably than by proximity-based methods, and (ii) discovery of interpretable directions, which correspond to semantic attributes such as age, ethnicity, or attire. Across four state-of-the-art recognition models (ArcFace, CosFace, ElasticFace, PartialFC) and two benchmarks (RFW, CelebA), LFA consistently outperforms k-means and nearest-neighbor search in intra-group semantic coherence, while uncovering interpretable latent directions aligned with demographic and contextual attributes. These results position LFA as a practical method for representation auditing of face recognition models, enabling practitioners to identify and interpret biased subpopulations without predefined attribute annotations.
Abstract (translated)
现代面部识别模型在整体准确性方面表现出色，但仍存在系统性偏见，这些偏见对某些特定亚群体的影响尤为显著。传统的偏差评估框架依赖于带有标签的属性来划分亚群，而获得这样的标签既昂贵又局限于预定义的类别范围之内。我们引入了一种名为隐含特征对齐（Latent Feature Alignment, LFA）的新算法，它无需标注属性即可通过隐含方向识别出亚群体。这种方法相较于标准聚类方法主要具有两大优势： (i) 半语义一致分组：LFA 能够更可靠地将共享相似属性的面孔进行分组，而不仅仅是基于位置接近性的方法。 (ii) 发现可解释的方向：该算法能够揭示与年龄、种族或着装等语义特征相对应的隐含方向。 在四个最先进的面部识别模型（ArcFace, CosFace, ElasticFace, PartialFC）以及两个基准测试数据集（RFW和CelebA）上，LFA 无论是在组内语义一致性还是发现与人口统计学和上下文属性相匹配的可解释性隐含方向方面均优于 k-means 聚类方法及最近邻搜索。 这些结果表明 LFA 可以作为一种实用的方法来审计面部识别模型的表现形式，帮助从业人员在没有预定义属性注释的情况下识别并解读有偏见的亚群体。
URL
https://arxiv.org/abs/2510.15520
PDF
-
QCFace: Image Quality Control for boosting Face Representation & Recognition
2025-10-17 04:00:20Duc-Phuong Doan-Ngo, Thanh-Dang Diep, Thanh Nguyen-Duc, Thanh-Sach LE, Nam ThoaiAbstract
Recognizability, a key perceptual factor in human face processing, strongly affects the performance of face recognition (FR) systems in both verification and identification tasks. Effectively using recognizability to enhance feature representation remains challenging. In deep FR, the loss function plays a crucial role in shaping how features are embedded. However, current methods have two main drawbacks: (i) recognizability is only partially captured through soft margin constraints, resulting in weaker quality representation and lower discrimination, especially for low-quality or ambiguous faces; (ii) mutual overlapping gradients between feature direction and magnitude introduce undesirable interactions during optimization, causing instability and confusion in hypersphere planning, which may result in poor generalization, and entangled representations where recognizability and identity are not cleanly separated. To address these issues, we introduce a hard margin strategy - Quality Control Face (QCFace), which overcomes the mutual overlapping gradient problem and enables the clear decoupling of recognizability from identity representation. Based on this strategy, a novel hard-margin-based loss function employs a guidance factor for hypersphere planning, simultaneously optimizing for recognition ability and explicit recognizability representation. Extensive experiments confirm that QCFace not only provides robust and quantifiable recognizability encoding but also achieves state-of-the-art performance in both verification and identification benchmarks compared to existing recognizability-based losses.
Abstract (translated)
可识别性是人类面部处理中的一个关键感知因素，它强烈影响着人脸验证和识别任务中人脸识别（FR）系统的性能。有效利用可识别性来增强特征表示仍然是一个挑战。在深度人脸识别中，损失函数在塑造特征如何嵌入方面起着至关重要的作用。然而，当前的方法存在两个主要缺点：(i) 可识别性仅通过软边距约束部分捕获，导致质量较低的表示和区分度较差，特别是在处理低质量和模糊人脸时；(ii) 特征方向与幅度之间的相互重叠梯度在优化过程中引入了不希望看到的交互作用，这可能导致泛化能力差，并且可识别性和身份特征纠缠不清。为了解决这些问题，我们提出了一种硬边距策略——质量控制面部（QCFace），该方法克服了相互重叠梯度的问题，并使得可识别性与身份表示能够清晰分离。基于这一策略，一种新的以硬边距为基础的损失函数采用了一个用于超球体规划的引导因子，同时优化识别人脸的能力和明确的可识别性表示。广泛的实验确认QCFace不仅提供了稳健且量化的可识别性编码，而且在验证和识别基准测试中相对于现有的基于可识别性的损失方法实现了最先进的性能。
URL
https://arxiv.org/abs/2510.15289
PDF
-
Benchmarking Multimodal Large Language Models for Face Recognition
2025-10-16 16:42:27Hatef Otroshi Shahreza, S\'ebastien MarcelAbstract
Multimodal large language models (MLLMs) have achieved remarkable performance across diverse vision-and-language tasks. However, their potential in face recognition remains underexplored. In particular, the performance of open-source MLLMs needs to be evaluated and compared with existing face recognition models on standard benchmarks with similar protocol. In this work, we present a systematic benchmark of state-of-the-art MLLMs for face recognition on several face recognition datasets, including LFW, CALFW, CPLFW, CFP, AgeDB and RFW. Experimental results reveal that while MLLMs capture rich semantic cues useful for face-related tasks, they lag behind specialized models in high-precision recognition scenarios in zero-shot applications. This benchmark provides a foundation for advancing MLLM-based face recognition, offering insights for the design of next-generation models with higher accuracy and generalization. The source code of our benchmark is publicly available in the project page.
Abstract (translated)
多模态大型语言模型（MLLM）在多种视觉和语言任务中取得了显著的性能。然而，它们在人脸识别领域的潜力尚未得到充分探索。特别是，开源的多模态大模型需要在其与现有标准基准测试相匹配的情况下进行评估，并与其他面部识别模型进行比较。在这项工作中，我们对几种面部识别数据集（包括LFW、CALFW、CPLFW、CFP、AgeDB和RFW）上最先进的MLLM进行了系统性的基准测试。实验结果表明，虽然多模态大模型能够捕捉到用于人脸相关任务的丰富语义线索，但在零样本应用中的高精度识别场景中，它们仍然落后于专门的人脸识别模型。这项基准测试为推进基于多模态大型语言模型的人脸识别奠定了基础，并为下一代更高准确性和泛化能力的模型设计提供了见解。我们的基准测试源代码已在项目页面上公开提供。
URL
https://arxiv.org/abs/2510.14866
PDF
