Abstract
The majority of inhabitants in Hong Kong are able to read and write in standard Chinese but use Cantonese as the primary spoken language in daily life. Spoken Cantonese can be transcribed into Chinese characters, which constitute the so-called written Cantonese. Written Cantonese exhibits significant lexical and grammatical differences from standard written Chinese. The rise of written Cantonese is increasingly evident in the cyber world. The growing interaction between Mandarin speakers and Cantonese speakers is leading to a clear demand for automatic translation between Chinese and Cantonese. This paper describes a transformer-based neural machine translation (NMT) system for written-Chinese-to-written-Cantonese translation. Given that parallel text data of Chinese and Cantonese are extremely scarce, a major focus of this study is on the effort of preparing good amount of training data for NMT. In addition to collecting 28K parallel sentences from previous linguistic studies and scattered internet resources, we devise an effective approach to obtaining 72K parallel sentences by automatically extracting pairs of semantically similar sentences from parallel articles on Chinese Wikipedia and Cantonese Wikipedia. We show that leveraging highly similar sentence pairs mined from Wikipedia improves translation performance in all test sets. Our system outperforms Baidu Fanyi's Chinese-to-Cantonese translation on 6 out of 8 test sets in BLEU scores. Translation examples reveal that our system is able to capture important linguistic transformations between standard Chinese and spoken Cantonese.
Abstract (translated)
香港的大多数居民能够阅读和书写标准汉语，但在日常生活中主要使用粤语作为口语。粤语可以通过汉字转写成所谓的“书面粤语”。书面粤语在词汇和语法上与标准书面中文有显著差异。随着网络世界的不断发展，书面粤语的应用越来越明显。普通话使用者和粤语使用者之间的互动日益增加，这导致了对中文与粤语文本自动翻译的明确需求。本文描述了一种基于Transformer的神经机器翻译（NMT）系统，用于将书面中文翻译成书面粤文。 鉴于汉语文本和粤语文本的平行数据极为稀少，此研究的主要焦点在于为NMT准备大量训练数据的努力上。除了从先前的语言学研究和散落于互联网上的资源中收集28,000个平行句子外，我们还设计了一种有效的方法，通过自动提取来自中文维基百科和粤语维基百科的并行文章中的语义相似句子对来获得72,000个平行句子。结果显示，利用从维基百科中挖掘出的高度相似句对能提高所有测试集上的翻译性能。 我们的系统在BLEU评分上，在8组测试集中有6组超越了百度翻译的中文到粤文翻译效果。翻译示例表明，该系统能够捕捉到标准汉语与口语粤语之间的重要的语言转换。
URL
https://arxiv.org/abs/2505.17816
