Abstract
This article concerns comparative studies on the Automatic Speech Recognition (ASR) model incorporated with the Large Language Model (LLM) used for medical interviews. The proposed solution is tested on polish language benchmarks and dataset with medical interviews. The latest ASR technologies are based on convolutional neural networks (CNNs), recurrent neural networks (RNNs) and Transformers. Most of them work as end-to-end solutions. The presented approach in the case of the Whisper model shows a two-stage solution with End-To-End ASR and LLM working together in a pipeline. The ASR output is an input for LLM. The LLM is a component by which the output from ASR is corrected and improved. Comparative studies for automatic recognition of the Polish language between modern End-To-End deep learning architectures and the ASR hybrid model were performed. The medical interview tests were performed with two state-of-the-art ASR models: OpenAI Whisper incorporated with LLM and Scribe ElevenLabs. Additionally, the results were compared with five more end-to-end models (QuartzNet, FastConformer, Wav2Vec 2.0 XLSR and ESPnet Model Zoo) on Mozilla Common Voice and VoxPopuli databases. Tests were conducted for clean audio signal, signal with bandwidth limitation, and degraded. The tested models were evaluated on the basis of Word Error Rate (WER) and Character Error Rate (CER). The results show that the Whisper model performs by far the best among the open-source models. ElevenLabs Scribe model, on the other hand, performs best for Polish on both general benchmark and medical data.
Abstract (translated)
本文探讨了将大型语言模型（LLM）集成到自动语音识别（ASR）模型中，特别是在医疗访谈场景中的比较研究。所提出的解决方案在波兰语基准测试和包含医学访谈的语料库上进行了测试。最新的ASR技术基于卷积神经网络（CNN）、循环神经网络（RNN）和Transformer架构，大多数实现为端到端解决方案。文中提出的方法对于Whisper模型则展示了一个两阶段方法：一个是由端到端ASR系统生成输出，并由LLM进行修正和优化的管道流程。 针对波兰语自动识别，在现代端到端深度学习架构与基于混合模型的ASR之间进行了比较研究。医疗访谈测试使用了两种前沿的ASR模型：OpenAI的Whisper模型结合LLM和ElevenLabs的Scribe ElevenLabs模型。同时，这些结果还与其他五种端到端模型（QuartzNet、FastConformer、Wav2Vec 2.0 XLSR及ESPnet Model Zoo）在Mozilla Common Voice和VoxPopuli数据库上的性能进行了比较。 测试涵盖了无噪声音频信号、带宽限制的信号以及退化的信号。评估这些模型的表现基于单词错误率（WER）和字符错误率（CER）。结果显示，开源模型中Whisper表现最佳；而ElevenLabs Scribe模型在波兰语通用基准测试及医疗数据上表现出色。
URL
https://arxiv.org/abs/2603.02246
