Abstract
Emotional Validation is a psychotherapy communication technique that involves recognizing, understanding, and explicitly acknowledging another person's feelings and actions, which strengthens alliance and reduces negative affect. To maximize the emotional support provided by validation, it is crucial to deliver it with appropriate timing and frequency. This study investigates validation timing detection from the speech perspective. Leveraging both paralinguistic and emotional information, we propose a paralinguistic- and emotion-aware model for validation timing detection without relying on textual context. Specifically, we first conduct continued self-supervised training and fine-tuning on different HuBERT backbones to obtain (i) a paralinguistics-aware Self-Supervised Learning (SSL) encoder and (ii) a multi-task speech emotion classification encoder. We then fuse these encoders and further fine-tune the combined model on the downstream validation timing detection task. Experimental evaluations on the TUT Emotional Storytelling Corpus (TESC) compare multiple models, fusion mechanisms, and training strategies, and demonstrate that the proposed approach achieves significant improvements over conventional speech baselines. Our results indicate that non-linguistic speech cues, when integrated with affect-related representations, carry sufficient signal to decide when validation should be expressed, offering a speech-first pathway toward more empathetic human-robot interaction.
Abstract (translated)
情感确认是心理疗法中的一个沟通技巧，涉及识别、理解和明确承认他人的情感和行为，这有助于加强关系并减少负面情绪。为了最大限度地提高情感支持的效果，适时且频繁地提供情感确认至关重要。本研究从语音角度探讨了情感确认时机的检测问题。通过利用副语言和情感信息，我们提出了一种不需要文本上下文、结合副语言感知与情感意识的情感确认时机检测模型。 具体来说，首先在不同的HuBERT骨干网络上进行持续自监督训练和微调，以获得（i）一种具有副语言感知能力的自监督学习(SSL)编码器以及(ii)一个多任务语音情绪分类编码器。然后将这些编码器融合，并进一步对组合模型进行下游情感确认时机检测任务的微调。 在TUT情感叙事语料库(TESC)上进行了多项实验评估，比较了多种模型、融合机制和训练策略的效果，表明所提出的方法在传统语音基线模型上有显著改进。我们的结果表明，当非语言性语音线索与情绪相关的表示结合时，足以决定何时表达确认信息，为实现更富有同理心的人机互动提供了一种以语音为主导的路径。
URL
https://arxiv.org/abs/2603.09307
