Abstract
Self-supervised speech models have demonstrated impressive performance in speech processing, but their effectiveness on non-speech data remains underexplored. We study the transfer learning capabilities of such models on bioacoustic detection and classification tasks. We show that models such as HuBERT, WavLM, and XEUS can generate rich latent representations of animal sounds across taxa. We analyze the models properties with linear probing on time-averaged representations. We then extend the approach to account for the effect of time-wise information with other downstream architectures. Finally, we study the implication of frequency range and noise on performance. Notably, our results are competitive with fine-tuned bioacoustic pre-trained models and show the impact of noise-robust pre-training setups. These findings highlight the potential of speech-based self-supervised learning as an efficient framework for advancing bioacoustic research.
Abstract (translated)
自监督语音模型在语音处理中表现出令人印象深刻的表现，但它们在非语音数据上的有效性仍需进一步探索。我们研究了这些模型在生物声学检测和分类任务中的迁移学习能力。研究表明，如HuBERT、WavLM和XEUS等模型能够生成跨物种的动物声音的丰富潜在表示。我们通过时间平均表示进行线性探测来分析模型特性，并随后扩展方法以考虑时序信息的影响及其对下游架构性能的作用。最后，我们研究了频率范围和噪声对性能的影响。值得注意的是，我们的结果与经过生物声学预训练并微调后的模型表现相当，并显示了鲁棒去噪预训练设置的重要性。这些发现突显了基于语音的自监督学习作为推进生物声学研究的有效框架的巨大潜力。
URL
https://arxiv.org/abs/2509.04166
