Abstract
Self-supervised learning has emerged as a powerful way to pre-train generalizable machine learning models on large amounts of unlabeled data. It is particularly compelling in the music domain, where obtaining labeled data is time-consuming, error-prone, and ambiguous. During the self-supervised process, models are trained on pretext tasks, with the primary objective of acquiring robust and informative features that can later be fine-tuned for specific downstream tasks. The choice of the pretext task is critical as it guides the model to shape the feature space with meaningful constraints for information encoding. In the context of music, most works have relied on contrastive learning or masking techniques. In this study, we expand the scope of pretext tasks applied to music by investigating and comparing the performance of new self-supervised methods for music tagging. We open-source a simple ResNet model trained on a diverse catalog of millions of tracks. Our results demonstrate that, although most of these pre-training methods result in similar downstream results, contrastive learning consistently results in better downstream performance compared to other self-supervised pre-training methods. This holds true in a limited-data downstream context.
Abstract (translated)
自监督学习已成为一种在大量未标注数据上预训练具有泛化机器学习模型的强大方法。在音乐领域，获得标注数据耗时、错误率高且具有不确定性。在自监督过程中，模型通过预处理任务进行训练，主要目标是在后续特定下游任务中获取稳健和有用的特征。预处理任务的选取对模型将如何对特征空间进行有意义的约束从而进行信息编码起着关键作用。在音乐领域，大多数作品都依赖对比学习或遮盖技术。在这项研究中，我们通过研究并比较新 Self-supervised 方法的音乐标签性能，扩展了应用于音乐的预处理任务的范畴。我们开源了一个基于多样化音乐目录的简单 ResNet 模型。我们的结果表明，尽管大多数预训练方法产生了类似的结果，但对比学习在下游性能方面始终优于其他 Self-supervised 预训练方法。在有限数据下游环境中，这一结论同样成立。
URL
https://arxiv.org/abs/2404.09177
