Abstract
This study examines pitch contours as a unifying semantic construct prevalent across various audio domains including music, speech, bioacoustics, and everyday sounds. Analyzing pitch contours offers insights into the universal role of pitch in the perceptual processing of audio signals and contributes to a deeper understanding of auditory mechanisms in both humans and animals. Conventional pitch-tracking methods, while optimized for music and speech, face challenges in handling much broader frequency ranges and more rapid pitch variations found in other audio domains. This study introduces a vision-based approach to pitch contour analysis that eliminates the need for explicit pitch-tracking. The approach uses a convolutional neural network, pre-trained for object detection in natural images and fine-tuned with a dataset of synthetically generated pitch contours, to extract key contour parameters from the time-frequency representation of short audio segments. A diverse set of eight downstream tasks from four audio domains were selected to provide a challenging evaluation scenario for cross-domain pitch contour analysis. The results show that the proposed method consistently surpasses traditional techniques based on pitch-tracking on a wide range of tasks. This suggests that the vision-based approach establishes a foundation for comparative studies of pitch contour characteristics across diverse audio domains.
Abstract (translated)
这项研究探讨了音高轮廓作为一种普遍的语义结构，它贯穿于音乐、语音、生物声学和日常声音等多种音频领域。分析音高轮廓能够提供关于音高在感知处理音频信号中的通用作用的见解，并有助于加深对人类和动物听觉机制的理解。传统音高跟踪方法虽然针对音乐和语音进行了优化，但在应对其他音频领域中存在的更宽泛频率范围以及更快的音高变化时面临挑战。 该研究提出了一种基于视觉的方法来分析音高轮廓，这种方法消除了显式音高追踪的需求。这种方案利用了预先训练好的用于自然图像物体检测的卷积神经网络，并通过合成生成的音高轮廓数据集进行微调，从短音频片段的时间-频率表示中提取关键轮廓参数。 为了对跨领域音高轮廓分析提供一个具有挑战性的评估场景，选择了来自四个不同音频领域的八项下游任务。研究结果表明，所提出的方法在一系列广泛的任务上持续优于基于传统音高跟踪的技术。这暗示着视觉方法为比较研究提供了基础，即探讨多种音频领域中音高轮廓特征的特性。
URL
https://arxiv.org/abs/2503.19161
