Abstract
Large Audio Language Models (LALMs) achieve strong performance on audio-language tasks; however, their reliability in real-world settings remains underexplored. We introduce Audio Hallucination Attacks (AHA), an attack suite called AHA-Eval, comprising 6.5K QA pairs designed to test whether LALMs genuinely ground their responses in the audio input. AHA targets two attack surfaces: (i) query-based attacks, which exploit question structure to induce hallucinations about absent sounds, and (ii) audio-based attacks, which inject synthetic speech describing non-existent events into the audio stream. Evaluating state-of-the-art LALMs, including Audio Flamingo 3 and Gemini 3 Pro, we observe high attack success rates of 95.35% and 79.65%, respectively, revealing a reliability gap that is hidden by standard benchmark performance. To mitigate this, we propose a 120K QA post-alignment dataset, AHA-Guard, which successfully reduces attack success rates by up to 49%.
Abstract (translated)
大型音频语言模型（LALMs）在音频-语言任务上表现出色；然而，其在真实场景中的可靠性尚未得到充分探索。我们提出了音频幻觉攻击（AHA）及其评估套件AHA-Eval，该套件包含6500组问答对，用于测试LALMs是否真正基于音频输入生成响应。AHA针对两种攻击面：（i）基于查询的攻击，利用问题结构诱使模型对不存在的音频产生幻觉；（ii）基于音频的攻击，向音频流中注入描述虚构事件的合成语音。在对包括Audio Flamingo 3和Gemini 3 Pro在内的先进LALMs进行评估时，我们观察到其攻击成功率分别高达95.35%和79.65%，揭示了标准基准测试所掩盖的可靠性差距。为缓解此问题，我们提出了一个包含12万组问答的后对齐数据集AHA-Guard，该数据集成功将攻击成功率最高降低了49%。
URL
https://arxiv.org/abs/2603.29263
