Abstract
Speech emotion recognition is crucial in human-computer interaction, but extracting and using emotional cues from audio poses challenges. This paper introduces MFHCA, a novel method for Speech Emotion Recognition using Multi-Spatial Fusion and Hierarchical Cooperative Attention on spectrograms and raw audio. We employ the Multi-Spatial Fusion module (MF) to efficiently identify emotion-related spectrogram regions and integrate Hubert features for higher-level acoustic information. Our approach also includes a Hierarchical Cooperative Attention module (HCA) to merge features from various auditory levels. We evaluate our method on the IEMOCAP dataset and achieve 2.6\% and 1.87\% improvements on the weighted accuracy and unweighted accuracy, respectively. Extensive experiments demonstrate the effectiveness of the proposed method.
Abstract (translated)
语音情感识别在人与计算机交互中至关重要，但提取和利用音频中的情感线索仍然具有挑战性。本文介绍了一种名为MFHCA的新方法，用于基于多空间融合和层次合作注意的语音情感识别。我们采用多空间融合模块（MF）来有效地识别与情感相关的频谱图区域，并利用Hubert特征获取更高层次的音频信息。我们的方法还包括一个层次合作注意模块（HCA），以合并来自不同音频层次的特征。我们在IEMOCAP数据集上评估我们的方法，分别实现了2.6%和1.87%的加权准确性和无加权准确性的提高。大量实验证明所提出的方法的有效性。
URL
https://arxiv.org/abs/2404.13509
