Abstract
Large Vision Language Models (LVLMs) have demonstrated remarkable capabilities across various downstream tasks, including biometric face recognition (FR) with description. However, demographic biases remain a critical concern in FR, as these foundation models often fail to perform equitably across diverse demographic groups, considering ethnicity/race, gender, and age. Therefore, through our work DemoBias, we conduct an empirical evaluation to investigate the extent of demographic biases in LVLMs for biometric FR with textual token generation tasks. We fine-tuned and evaluated three widely used pre-trained LVLMs: LLaVA, BLIP-2, and PaliGemma on our own generated demographic-balanced dataset. We utilize several evaluation metrics, like group-specific BERTScores and the Fairness Discrepancy Rate, to quantify and trace the performance disparities. The experimental results deliver compelling insights into the fairness and reliability of LVLMs across diverse demographic groups. Our empirical study uncovered demographic biases in LVLMs, with PaliGemma and LLaVA exhibiting higher disparities for Hispanic/Latino, Caucasian, and South Asian groups, whereas BLIP-2 demonstrated comparably consistent. Repository: this https URL.
Abstract (translated)
大型视觉语言模型（LVLM）在包括带描述的生物识别面部识别（FR）在内的各种下游任务中表现出色。然而，人口统计学偏见仍然是面部识别中的一个关键问题，因为这些基础模型往往无法在考虑种族/民族、性别和年龄等多样化的群体时实现公平性。因此，在我们的工作DemoBias中，我们通过实证评估来调查LVLM在生物特征FR任务中文本标记生成方面的种族偏见程度。我们在自动生成的均衡人口统计学数据集上对三个广泛使用的预训练LVLM进行了微调和评估：LLaVA、BLIP-2和PaliGemma。我们使用了多种评估指标，如特定群体的BERT得分和公平性差异率，以量化和追踪性能差距。实验结果为LVLM在不同人口统计学群体中的公平性和可靠性提供了深刻的见解。我们的实证研究发现，在LLaVA和PaliGemma中，西班牙裔/拉丁裔、高加索人种和南亚人群体之间的偏见程度较高，而BLIP-2则表现出相对一致的表现。 相关代码库链接：[此URL](this https URL)
URL
https://arxiv.org/abs/2508.19298
