Abstract
Speech Large Language Models (LLMs) show great promise for speech emotion recognition (SER) via generative interfaces. However, shifting from closed-set classification to open text generation introduces zero-shot stochasticity, making evaluation highly sensitive to prompts. Additionally, conventional speech LLMs benchmarks overlook the inherent ambiguity of human emotion. Hence, we present VoxEmo, a comprehensive SER benchmark encompassing 35 emotion corpora across 15 languages for Speech LLMs. VoxEmo provides a standardized toolkit featuring varying prompt complexities, from direct classification to paralinguistic reasoning. To reflect real-world perception/application, we introduce a distribution-aware soft-label protocol and a prompt-ensemble strategy that emulates annotator disagreement. Experiments reveal that while zero-shot speech LLMs trail supervised baselines in hard-label accuracy, they uniquely align with human subjective distributions.
Abstract (translated)
语音大型语言模型（LLMs）在通过生成接口进行语音情感识别（SER）方面显示出巨大潜力。然而，从封闭集分类转向开放文本生成引入了零样本随机性，使得评估高度依赖于提示的敏感度增加。此外，传统的语音LLM基准测试忽略了人类情绪固有的模糊性。因此，我们提出了VoxEmo，这是一个全面的SER基准测试工具，涵盖了15种语言中的35个情感语料库，专为语音LLMs设计。VoxEmo提供了一个标准化的工具包，其中包括从直接分类到旁语义推理的各种提示复杂度。 为了反映真实世界的应用感知情况，我们引入了一种分布感知的软标签协议和一种提示集成策略，该策略模拟了标注人员之间的分歧意见。实验表明，虽然零样本语音LLMs在硬标签准确性上落后于监督基线模型，但它们独特地与人类主观分布相一致。
URL
https://arxiv.org/abs/2603.08936
