Abstract
Automatic lyrics transcription (ALT) remains a challenging task in the field of music information retrieval, despite great advances in automatic speech recognition (ASR) brought about by transformer-based architectures in recent years. One of the major challenges in ALT is the high amplitude of interfering audio signals relative to conventional ASR due to musical accompaniment. Recent advances in music source separation have enabled automatic extraction of high-quality separated vocals, which could potentially improve ALT performance. However, the effect of source separation has not been systematically investigated in order to establish best practices for its use. This work examines the impact of source separation on ALT using Whisper, a state-of-the-art open source ASR model. We evaluate Whisper's performance on original audio, separated vocals, and vocal stems across short-form and long-form transcription tasks. For short-form, we suggest a concatenation method that results in a consistent reduction in Word Error Rate (WER). For long-form, we propose an algorithm using source separation as a vocal activity detector to derive segment boundaries, which results in a consistent reduction in WER relative to Whisper's native long-form algorithm. Our approach achieves state-of-the-art results for an open source system on the Jam-ALT long-form ALT benchmark, without any training or fine-tuning. We also publish MUSDB-ALT, the first dataset of long-form lyric transcripts following the Jam-ALT guidelines for which vocal stems are publicly available.
Abstract (translated)
自动歌词转录（ALT）在音乐信息检索领域仍然是一个具有挑战性的任务，尽管近年来基于变压器架构的自动语音识别（ASR）技术取得了重大进展。ALT的一个主要挑战是由于音乐伴奏的存在，干扰音频信号相对于传统ASR来说幅度更大。最近在音乐源分离领域的进步使得可以从原始音频中提取高质量的分离人声，这有可能提高ALT性能。然而，关于源分离的效果尚未系统地进行研究以建立最佳实践方法。 这项工作利用Whisper（一种最先进的开源ASR模型），考察了源分离对ALT的影响。我们在短形式和长形式转录任务上评估了Whisper在原始音频、分离人声以及人声音轨上的表现。对于短形式，我们提出了一种拼接方法，这种方法可以持续降低词错误率（WER）。对于长形式，我们建议使用源分离作为音效检测器来推导片段边界的方法，相比起Whisper原生的长形式算法，这种方法能一致地减少WER。 我们的方法在不进行任何训练或微调的情况下，在Jam-ALT长形式ALT基准测试中达到了开源系统的最新水平。此外，我们发布了MUSDB-ALT数据集，这是首个遵循Jam-ALT指南的长形式歌词转录数据集，并公开了人声音轨。
URL
https://arxiv.org/abs/2506.15514
