Abstract
Deep speech classification tasks, including keyword spotting and speaker verification, are vital in speech-based human-computer interaction. Recently, the security of these technologies has been revealed to be susceptible to backdoor attacks. Specifically, attackers use noisy disruption triggers and speech element triggers to produce poisoned speech samples that train models to become vulnerable. However, these methods typically create only a limited number of backdoors due to the inherent constraints of the trigger function. In this paper, we propose that speech backdoor attacks can strategically focus on speech elements such as timbre and emotion, leveraging the Speech Large Language Model (SLLM) to generate diverse triggers. Increasing the number of triggers may disproportionately elevate the poisoning rate, resulting in higher attack costs and a lower success rate per trigger. We introduce the Multiple Gradient Descent Algorithm (MGDA) as a mitigation strategy to address this challenge. The proposed attack is called the Speech Prompt Backdoor Attack (SPBA). Building on this foundation, we conducted attack experiments on two speech classification tasks, demonstrating that SPBA shows significant trigger effectiveness and achieves exceptional performance in attack metrics.
Abstract (translated)
深度语音分类任务，包括关键词识别和说话人验证，在基于语音的人机交互中至关重要。然而，最近的研究揭示了这些技术的安全性容易受到后门攻击的威胁。具体来说，攻击者利用噪声干扰触发器和语音元素触发器来生成中毒语音样本，使模型变得脆弱。不过，由于触发函数本身的限制，这些方法通常只能创建有限数量的后门。 在本文中，我们提出了一种策略，即通过专注于语音中的音色和情感等元素，并借助于语音大语言模型（SLLM）生成多样化的触发器来实施语音后门攻击。增加触发器的数量可能会不成比例地提高中毒率，导致攻击成本上升而每个触发器的成功率降低。 为了应对这一挑战，我们引入了多重梯度下降算法（MGDA）作为缓解策略，并提出了名为“Speech Prompt Backdoor Attack”（SPBA）的新型攻击方法。基于此基础，我们在两个语音分类任务上进行了攻击实验，结果表明SPBA展示了显著的触发器有效性，并在攻击指标方面取得了卓越性能。 该研究的主要贡献包括： 1. 提出了一种新的后门攻击策略，利用多样化的语音元素来增加模型中毒的可能性。 2. 引入了MGDA作为缓解方法，旨在减少此类攻击的影响。 3. 通过实验验证了SPBA的有效性和效率，在实际应用中具有重要的安全意义。
URL
https://arxiv.org/abs/2506.08346
