Abstract
Face Recognition Systems (FRS) are increasingly vulnerable to face-morphing attacks, prompting the development of Morphing Attack Detection (MAD) algorithms. However, a key challenge in MAD lies in its limited generalizability to unseen data and its lack of explainability-critical for practical application environments such as enrolment stations and automated border control systems. Recognizing that most existing MAD algorithms rely on supervised learning paradigms, this work explores a novel approach to MAD using zero-shot learning leveraged on Large Language Models (LLMs). We propose two types of zero-shot MAD algorithms: one leveraging general vision models and the other utilizing multimodal LLMs. For general vision models, we address the MAD task by computing the mean support embedding of an independent support set without using morphed images. For the LLM-based approach, we employ the state-of-the-art GPT-4 Turbo API with carefully crafted prompts. To evaluate the feasibility of zero-shot MAD and the effectiveness of the proposed methods, we constructed a print-scan morph dataset featuring various unseen morphing algorithms, simulating challenging real-world application scenarios. Experimental results demonstrated notable detection accuracy, validating the applicability of zero-shot learning for MAD tasks. Additionally, our investigation into LLM-based MAD revealed that multimodal LLMs, such as ChatGPT, exhibit remarkable generalizability to untrained MAD tasks. Furthermore, they possess a unique ability to provide explanations and guidance, which can enhance transparency and usability for end-users in practical applications.
Abstract (translated)
面部识别系统（FRS）日益容易受到面部合成攻击的威胁，从而推动了用于检测这些攻击的变形攻击检测（MAD）算法的发展。然而，MAD的一个关键挑战在于其有限的泛化能力以及在实际应用环境中的缺乏可解释性，例如注册站和自动边境控制系统。鉴于大多数现有的MAD算法依赖于监督学习范式，本研究探索了一种新的基于零样本学习的方法来开发MAD算法，并利用大型语言模型（LLMs）。我们提出两种类型的零样本MAD算法：一种是依靠通用视觉模型，另一种则使用多模态LLMs。 对于通用视觉模型，我们在不使用变形图像的情况下通过计算独立支持集的平均支持嵌入来解决MAD任务。在基于LLMs的方法中，我们采用最先进的GPT-4 Turbo API，并结合精心设计的提示语进行操作。为了评估零样本MAD的可行性以及所提出方法的有效性，我们构建了一个打印扫描变形数据集，该数据集中包含了各种未见过的变形算法，模拟了具有挑战性的现实应用场景。实验结果表明，在检测准确性方面取得了显著成就，验证了零样本学习在MAD任务中的适用性。 此外，我们的研究发现基于LLMs的MAD方法表现出色：多模态LLMs（如ChatGPT）对未训练过的MAD任务展示出了出色的泛化能力，并且它们还具备提供解释和指导的独特能力，这可以增强实际应用中最终用户透明度和可用性。
URL
https://arxiv.org/abs/2503.10937
