Abstract
Face recognition (FR) systems are often prone to demographic biases, partially due to the entanglement of demographic-specific information with identity-relevant features in facial embeddings. This bias is extremely critical in large multicultural cities, especially where biometrics play a major role in smart city infrastructure. The entanglement can cause demographic attributes to overshadow identity cues in the embedding space, resulting in disparities in verification performance across different demographic groups. To address this issue, we propose a novel strategy, Unified Text-Image Embedding (UTIE), which aims to induce demographic ambiguity in face embeddings by enriching them with information related to other demographic groups. This encourages face embeddings to emphasize identity-relevant features and thus promotes fairer verification performance across groups. UTIE leverages the zero-shot capabilities and cross-modal semantic alignment of Vision-Language Models (VLMs). Given that VLMs are naturally trained to align visual and textual representations, we enrich the facial embeddings of each demographic group with text-derived demographic features extracted from other demographic groups. This encourages a more neutral representation in terms of demographic attributes. We evaluate UTIE using three VLMs, CLIP, OpenCLIP, and SigLIP, on two widely used benchmarks, RFW and BFW, designed to assess bias in FR. Experimental results show that UTIE consistently reduces bias metrics while maintaining, or even improving in several cases, the face verification accuracy.
Abstract (translated)
人脸识别（FR）系统常常存在人口统计学偏差，部分原因是面部嵌入中包含了特定于某一人群的信息与身份相关特征的纠缠。这种偏见在大型多元文化城市尤其严重，尤其是在生物识别技术对智慧城市基础设施发挥重要作用的情况下。这种纠缠可能导致人口统计属性在嵌入空间中压倒了身份线索，从而导致不同人口群体之间的验证性能差异。为解决这一问题，我们提出了一种新颖策略——统一文本图像嵌入（UTIE），旨在通过将与其它人口群体相关的特征信息加入面部嵌入来诱导出人口统计学的模糊性。这鼓励面部嵌入强调身份相关特征，从而促进不同群体间更公平的验证性能。 UTIE利用了视觉语言模型（VLM）的零样本能力和跨模态语义对齐能力。由于VLM自然地训练用于调整视觉和文本表示的一致性，我们用从其它人口统计学群组提取出的文字衍生的人口特征来丰富每个群体的面部嵌入。这鼓励了一个更中立的人口属性表示。 我们在CLIP、OpenCLIP 和 SigLIP 三种 VLM 上使用 RFW 和 BFW 这两个广泛使用的基准进行UTIE评估，后者设计用于测试 FR 中的偏见问题。实验结果显示 UTIE 在降低偏见指标的同时保持甚至在某些情况下提高了面部验证准确率。
URL
https://arxiv.org/abs/2512.08981
