Abstract
Domain-specific speech remains a persistent challenge for automatic speech recognition (ASR), even for state-of-the-art systems like OpenAI's Whisper. We introduce Whisper: Courtside Edition, a novel multi-agent large language model (LLM) pipeline that enhances Whisper transcriptions without retraining. The pipeline intercepts Whisper's initial transcript, applies specialized LLM agents for domain context identification, named entity recognition, and jargon detection, and generates compact prompts that guide Whisper's decoder. Evaluated on 421 NBA basketball commentary segments (a domain characterized by dense proper nouns and technical terminology) our best pipeline achieves a statistically significant 17.0% relative reduction in word error rate (WER; from 0.217 to 0.180, p<0.001). Improvements are observed in 40.1% of segments with degradation in only 7.1%, substantially outperforming direct transcript post-editing. These results demonstrate that prompt-based augmentation can deliver scalable domain adaptation for ASR, offering a practical alternative to costly model fine-tuning.
Abstract (translated)
领域特定的语音识别仍然是自动语音识别（ASR）系统面临的一个持久挑战，即使对于像OpenAI的Whisper这样的最先进的系统也是如此。我们引入了Whisper：Courtside Edition，这是一个新颖的多代理大型语言模型（LLM）管道，可以在不重新训练的情况下增强Whisper的转录效果。该管道拦截Whisper生成的初始转录，并应用专门的LLM代理进行领域上下文识别、命名实体识别和行话检测，然后生成简短提示来指导Whisper解码器的工作。 在评估中，我们使用了421段NBA篮球评论（这一领域以密集的人名和其他专有名词以及技术术语为特征）测试我们的最佳管道。结果表明，在统计学上有显著意义的情况下，相对错误率降低了17.0%（从0.217降至0.180，p<0.001）。在40.1%的片段中观察到了改进，只有7.1%的片段出现退化现象。这远远优于直接对转录进行后编辑的效果。 这些结果表明基于提示的增强技术可以为ASR提供可扩展领域的适应性，并且作为一种替代方案，相对于成本高昂的模型微调来说更加实际可行。
URL
https://arxiv.org/abs/2602.18966
