Abstract
Multimodal Large Language Models (MLLMs) have recently demonstrated strong performance on a wide range of vision-language tasks, raising interest in their potential use for biometric applications. In this paper, we conduct a systematic evaluation of state-of-the-art MLLMs for heterogeneous face recognition (HFR), where enrollment and probe images are from different sensing modalities, including visual (VIS), near infrared (NIR), short-wave infrared (SWIR), and thermal camera. We benchmark multiple open-source MLLMs across several cross-modality scenarios, including VIS-NIR, VIS-SWIR, and VIS-THERMAL face recognition. The recognition performance of MLLMs is evaluated using biometric protocols and based on different metrics, including Acquire Rate, Equal Error Rate (EER), and True Accept Rate (TAR). Our results reveal substantial performance gaps between MLLMs and classical face recognition systems, particularly under challenging cross-spectral conditions, in spite of recent advances in MLLMs. Our findings highlight the limitations of current MLLMs for HFR and also the importance of rigorous biometric evaluation when considering their deployment in face recognition systems.
Abstract (translated)
最近，多模态大型语言模型（MLLMs）在广泛的视觉-语言任务中表现出色，引发了人们对其在生物识别应用中的潜在用途的兴趣。在这篇论文中，我们对最先进的MLLMs进行了系统评估，用于异构人脸识别（HFR），其中注册和探针图像来自不同的传感模式，包括可见光（VIS）、近红外（NIR）、短波红外（SWIR）和热成像相机。我们在多个跨模态场景下对多种开源MLLM进行基准测试，其中包括VIS-NIR、VIS-SWIR和VIS-THERMAL人脸识别。 我们使用生物识别协议并基于不同的指标，包括获取率(Acquire Rate)、等错误率(EER)和真正接受率(TAR)，评估了MLLMs的识别性能。我们的研究结果揭示了在具有挑战性的跨光谱条件下，即使是最近取得的进步，MLLMs与经典的人脸识别系统之间仍然存在显著的性能差距。这些发现强调了当前MLLMs在HFR中的局限性，并且也突显了在其部署于人脸识别系统时进行严格的生物特征评估的重要性。
URL
https://arxiv.org/abs/2601.15406
