Abstract
Automatic speech recognition systems have achieved remarkable performance on fluent speech but continue to degrade significantly when processing stuttered speech, a limitation that is particularly acute for low-resource languages like Indonesian where specialized datasets are virtually non-existent. To overcome this scarcity, we propose a data augmentation framework that generates synthetic stuttered audio by injecting repetitions and prolongations into fluent text through a combination of rule-based transformations and large language models followed by text-to-speech synthesis. We apply this synthetic data to fine-tune a pre-trained Indonesian Whisper model using transfer learning, enabling the architecture to adapt to dysfluent acoustic patterns without requiring large-scale real-world recordings. Our experiments demonstrate that this targeted synthetic exposure consistently reduces recognition errors on stuttered speech while maintaining performance on fluent segments, validating the utility of synthetic data pipelines for developing more inclusive speech technologies in under-represented languages.
Abstract (translated)
自动语音识别系统在处理流畅的语音时已经取得了显著的成绩，但在处理口吃语音时性能仍然大幅下降，这一局限性在印尼语等资源匮乏的语言中尤为严重，因为这些语言缺乏专门的数据集。为了克服这种稀缺问题，我们提出了一种数据增强框架，该框架通过结合规则基础变换和大型语言模型来生成合成的口吃音频，在流畅文本中插入重复和延长部分，随后再进行语音合成。我们将这一合成数据应用于使用迁移学习对预训练的印尼语Whisper模型进行微调，使架构能够在不依赖大规模真实世界录音的情况下适应口吃的声学模式。实验表明，这种针对性的合成暴露方法在减少口吃语音识别错误方面具有持续性效果，并且保持了对流畅部分的表现力，从而验证了合成数据管道在开发更包容的语言技术方面的实用性，尤其是在代表性不足的语言中。
URL
https://arxiv.org/abs/2601.03727
