Abstract
Transfer learning using latent representations from pre-trained speech models achieves outstanding performance in tasks where labeled data is scarce. However, their applicability to non-speech data and the specific acoustic properties encoded in these representations remain largely unexplored. In this study, we investigate both aspects. We evaluate five pre-trained speech models on eight baby cries datasets, encompassing 115 hours of audio from 960 babies. For each dataset, we assess the latent representations of each model across all available classification tasks. Our results demonstrate that the latent representations of these models can effectively classify human baby cries and encode key information related to vocal source instability and identity of the crying baby. In addition, a comparison of the architectures and training strategies of these models offers valuable insights for the design of future models tailored to similar tasks, such as emotion detection.
Abstract (translated)
使用预训练语音模型的潜在表示进行迁移学习，在标记数据稀缺的任务中表现出卓越性能。然而，这些方法在非语音数据中的适用性以及它们所编码的具体声学特性仍然很大程度上未被探索。在这项研究中，我们探讨了这两个方面。我们在八个婴儿哭声数据集上评估了五种预训练的语音模型，这些数据集中包含来自960个婴儿的总计115小时音频。对于每个数据集，我们对每种模型的所有可用分类任务进行了潜在表示的评估。 我们的研究结果表明，这些模型的潜在表示能够有效地对人类婴儿哭声进行分类，并且编码了与声音来源不稳定性和哭泣婴儿身份相关的关键信息。此外，对比分析这些模型的架构和训练策略为设计未来适用于类似任务（如情感检测）的模型提供了有价值的见解。
URL
https://arxiv.org/abs/2509.02259
