Abstract
Speech emotion recognition (SER) is constantly gaining attention in recent years due to its potential applications in diverse fields and thanks to the possibility offered by deep learning technologies. However, recent studies have shown that deep learning models can be vulnerable to adversarial attacks. In this paper, we systematically assess this problem by examining the impact of various adversarial white-box and black-box attacks on different languages and genders within the context of SER. We first propose a suitable methodology for audio data processing, feature extraction, and CNN-LSTM architecture. The observed outcomes highlighted the significant vulnerability of CNN-LSTM models to adversarial examples (AEs). In fact, all the considered adversarial attacks are able to significantly reduce the performance of the constructed models. Furthermore, when assessing the efficacy of the attacks, minor differences were noted between the languages analyzed as well as between male and female speech. In summary, this work contributes to the understanding of the robustness of CNN-LSTM models, particularly in SER scenarios, and the impact of AEs. Interestingly, our findings serve as a baseline for a) developing more robust algorithms for SER, b) designing more effective attacks, c) investigating possible defenses, d) improved understanding of the vocal differences between different languages and genders, and e) overall, enhancing our comprehension of the SER task.
Abstract (translated)
近年来，由于其在各种领域具有潜在应用以及深度学习技术的优势，情感识别（SER）引起了越来越多的关注。然而，最近的研究表明，深度学习模型可能容易受到对抗攻击。在本文中，我们通过研究各种对抗性白盒和黑盒攻击对不同语言和性别在SER背景下的影响，系统地评估了这个问题。我们首先提出了一个音频数据处理、特征提取和CNN-LSTM架构的合适方法。观察到的结果突出了CNN-LSTM模型对对抗实例（AEs）的重大漏洞。事实上，所考虑的所有攻击都能够显著地降低构建模型的性能。此外，在评估攻击的有效性时，分析的语言之间以及男性和女性之间的差异较小。总之，本工作为理解CNN-LSTM模型的SER鲁棒性以及AEs的影响做出了贡献。有趣的是，我们的研究为a)为SER开发更健壮的算法，b)设计更有效的攻击，c)研究可能的防御，d)改进对不同语言和性别之间语音差异的理解，以及e)提高对SER任务的全面理解做出了贡献。
URL
https://arxiv.org/abs/2404.18514
