Abstract
This paper presents and evaluates an optimized cascaded Nepali speech-to-English text translation (S2TT) system, focusing on mitigating structural noise introduced by Automatic Speech Recognition (ASR). We first establish highly proficient ASR and NMT components: a Wav2Vec2-XLS-R-300m model achieved a state-of-the-art 2.72% CER on OpenSLR-54, and a multi-stage fine-tuned MarianMT model reached a 28.32 BLEU score on the FLORES-200 benchmark. We empirically investigate the influence of punctuation loss, demonstrating that unpunctuated ASR output significantly degrades translation quality, causing a massive 20.7% relative BLEU drop on the FLORES benchmark. To overcome this, we propose and evaluate an intermediate Punctuation Restoration Module (PRM). The final S2TT pipeline was tested across three configurations on a custom dataset. The optimal configuration, which applied the PRM directly to ASR output, achieved a 4.90 BLEU point gain over the direct ASR-to-NMT baseline (BLEU 36.38 vs. 31.48). This improvement was validated by human assessment, which confirmed the optimized pipeline's superior Adequacy (3.673) and Fluency (3.804). This work validates that targeted punctuation restoration is the most effective intervention for mitigating structural noise in the Nepali S2TT pipeline. It establishes an optimized baseline and demonstrates a critical architectural insight for developing cascaded speech translation systems for similar low-resource languages.
Abstract (translated)
这篇论文提出并评估了一种优化的尼泊尔语语音到英文文本翻译（S2TT）系统，重点在于减少自动语音识别（ASR）引入的结构噪音。我们首先建立了高度熟练的ASR和神经机器翻译(NMT)组件：一个Wav2Vec2-XLS-R-300m模型在OpenSLR-54数据集上取得了最先进的2.72% CER（字符错误率），而一个多阶段微调的MarianMT模型在FLORES-200基准测试中达到了28.32 BLEU分。我们通过实证研究探讨了标点符号损失的影响，证明未经标点处理的ASR输出显著降低了翻译质量，在FLORES基准上导致了令人震惊的20.7%相对BLEU分数下降。为克服这一问题，我们提出并评估了一种中间的标点符号恢复模块（PRM）。最终的S2TT流水线在自定义数据集上的三种配置中进行了测试。最优配置是直接将PRM应用于ASR输出，比直接从ASR到NMT的基础模型取得了4.90个BLEU分的增长（36.38 BLEU vs 31.48 BLEU）。这一改进通过人工评估得到了验证，该评估确认了优化流水线在充分性(3.673)和流畅度(3.804)方面的优越表现。这项工作证实，针对标点符号的恢复是减轻尼泊尔语S2TT管道中结构噪音最有效的干预措施，并建立了优化基准并展示了开发类似低资源语言级联语音翻译系统的关键架构洞察力。
URL
https://arxiv.org/abs/2602.21647
