Abstract
Voice deepfake attacks, which artificially impersonate human speech for malicious purposes, have emerged as a severe threat. Existing defenses typically inject noise into human speech to compromise voice encoders in speech synthesis models. However, these methods degrade audio quality and require prior knowledge of the attack approaches, limiting their effectiveness in diverse scenarios. Moreover, real-time audios, such as speech in virtual meetings and voice messages, are still exposed to voice deepfake threats. To overcome these limitations, we propose ClearMask, a noise-free defense mechanism against voice deepfake attacks. Unlike traditional approaches, ClearMask modifies the audio mel-spectrogram by selectively filtering certain frequencies, inducing a transferable voice feature loss without injecting noise. We then apply audio style transfer to further deceive voice decoders while preserving perceived sound quality. Finally, optimized reverberation is introduced to disrupt the output of voice generation models without affecting the naturalness of the speech. Additionally, we develop LiveMask to protect streaming speech in real-time through a universal frequency filter and reverberation generator. Our experimental results show that ClearMask and LiveMask effectively prevent voice deepfake attacks from deceiving speaker verification models and human listeners, even for unseen voice synthesis models and black-box API services. Furthermore, ClearMask demonstrates resilience against adaptive attackers who attempt to recover the original audio signal from the protected speech samples.
Abstract (translated)
声纹深度伪造攻击，这种利用人工智能技术模仿人类语音以达到恶意目的的行为，已经成为了严重的威胁。现有的防御措施通常会在人的声音中加入噪声来破坏语音合成模型中的语音编码器。然而，这些方法会降低音频质量，并且需要事先了解攻击的手段，这在多变的情景下限制了它们的有效性。此外，在线实时音频（如虚拟会议和语音消息）仍然容易受到声纹深度伪造攻击。 为克服以上局限性，我们提出了ClearMask这一种无噪音防御机制来对抗声纹深度伪造攻击。与传统的做法不同，ClearMask通过选择性地过滤某些频率来修改音频的梅尔频谱图，从而在不注入噪声的情况下诱导出可转移的语音特征损失。之后，我们应用了音频风格迁移技术进一步欺骗语音解码器同时保持声音的质量不受影响。最后，引入优化后的混响处理以干扰声纹生成模型输出的结果而不影响自然说话的声音特性。 此外，为了保护实时流媒体中的讲话内容，我们开发了一种名为LiveMask的方法，通过通用频率滤波和回音发生器来实现这一目标。 实验结果表明，ClearMask和LiveMask在面对未知的语音合成模型以及黑盒API服务时，都能有效阻止声纹深度伪造攻击欺骗说话人验证模型及人类听者。此外，ClearMask还展现出了对那些试图从保护后的音频样本中恢复出原始信号的自适应攻击者的抗性。
URL
https://arxiv.org/abs/2508.17660
