Abstract
Speech Emotion Captioning (SEC) leverages large audio-language models to generate rich, context-aware affective descriptions from speech. However, real-world deployment remains challenging due to the substantial computational demands on resource-constrained edge devices and the privacy risks of transmitting biometric audio. While smaller audio-language models enable efficient on-device SEC, their limited capacity often weakens subtle paralinguistic modeling and fine-grained affective grounding. We propose an edge-cloud collaborative framework based on Uncertainty-Guided Speculative Decoding (UGSD). A lightweight edge model drafts captions locally, and only high-uncertainty token blocks are selectively escalated to a stronger cloud verifier for validation. Experiments on the MER2024 benchmark demonstrate substantial BLEU improvements up to 62.7%. UGSD further achieves 1.4x lower latency and 8.5x higher token throughput compared to an edge-only model. These results empirically characterize the quality-efficiency-privacy trade-off in deployable SEC systems.
Abstract (translated)
语音情感字幕（SEC）技术利用大型音频语言模型从语音中生成丰富且上下文感知的情感描述。然而，由于资源受限的边缘设备计算需求较大以及传输生物识别音频带来的隐私风险，其实用部署仍然面临挑战。虽然较小的音频语言模型能够在设备上高效实现SEC，但它们有限的能力往往削弱了细微的副语言建模和情感细粒度定位。 为解决这些问题，我们提出了一种基于不确定性引导推测解码（UGSD）的边缘-云协作框架。在这个框架中，轻量级的边缘模型在本地生成字幕，而只有具有高不确定性的标记块才会被选择性地提升到更强的云端验证器进行验证。我们在MER2024基准测试上的实验表明，BLEU评分提高了高达62.7%。此外，UGSD还实现了1.4倍更低的延迟和8.5倍更高的令牌吞吐量，相较于仅边缘模型的方法。这些结果实证地描述了部署SEC系统时质量、效率和隐私之间的权衡关系。 这种框架不仅增强了情感字幕的质量和生成速度，同时也降低了传输敏感数据的风险，为在资源受限环境中的实用部署提供了可能。
URL
https://arxiv.org/abs/2603.11397
