Abstract
The same speech content produced by different speakers exhibits significant differences in pitch contour, yet listeners' semantic perception remains unaffected. This phenomenon may stem from the brain's perception of pitch contours being independent of individual speakers' pitch ranges. In this work, we recorded electroencephalogram (EEG) while participants listened to Mandarin monosyllables with varying tones, phonemes, and speakers. The CE-ViViT model is proposed to decode raw or speaker-normalized pitch contours directly from EEG. Experimental results demonstrate that the proposed model can decode pitch contours with modest errors, achieving performance comparable to state-of-the-art EEG regression methods. Moreover, speaker-normalized pitch contours were decoded more accurately, supporting the neural encoding of relative pitch.
Abstract (translated)
不同说话者发出的相同演讲内容在音高曲线上表现出显著差异，但听众的语义感知并未受到影响。这一现象可能源于大脑对音高曲线的感知与个别说话人的音域独立。在这项工作中，我们记录了参与者聆听具有不同声调、音素和说话者的普通话单音节词时的脑电图（EEG）。提出了CE-ViViT模型来直接从EEG解码原始或说话人归一化的音高曲线。实验结果表明，所提出的模型可以以适度误差解码音高曲线，并且性能与最先进的EEG回归方法相当。此外，更准确地解码了说话人归一化的音高曲线，支持相对音高的神经编码。
URL
https://arxiv.org/abs/2505.19626
