Abstract
This work introduces {\it PrahokBART}, a compact pre-trained sequence-to-sequence model trained from scratch for Khmer using carefully curated Khmer and English corpora. We focus on improving the pre-training corpus quality and addressing the linguistic issues of Khmer, which are ignored in existing multilingual models, by incorporating linguistic components such as word segmentation and normalization. We evaluate PrahokBART on three generative tasks: machine translation, text summarization, and headline generation, where our results demonstrate that it outperforms mBART50, a strong multilingual pre-trained model. Additionally, our analysis provides insights into the impact of each linguistic module and evaluates how effectively our model handles space during text generation, which is crucial for the naturalness of texts in Khmer.
Abstract (translated)
这项工作介绍了{\it PrahokBART}，这是一种专门为高棉语从零开始训练的紧凑型预训练序列到序列模型，并使用精心整理的高棉语和英语语料库。我们专注于提高预训练语料库的质量并解决现有多语言模型忽略的高棉语的语言问题，通过引入诸如词分割和规范化等语言组件来实现这一目标。我们在三项生成任务上评估了PrahokBART：机器翻译、文本摘要和标题生成，在这些任务中，我们的结果表明它优于mBART50，这是一个强大的多语言预训练模型。此外，我们的分析提供了对每个语言模块影响的见解，并评估了在高棉语文本生成过程中我们模型处理空间的效果，这对提高文本自然度至关重要。
URL
https://arxiv.org/abs/2512.13552
