Abstract
Word meaning, representation, and interpretation play fundamental roles in natural language understanding (NLU), natural language processing (NLP), and natural language generation (NLG) tasks. Many of the inherent difficulties in these tasks stem from Multi-word Expressions (MWEs), which complicate the tasks by introducing ambiguity, idiomatic expressions, infrequent usage, and a wide range of variations. Significant effort and substantial progress have been made in addressing the challenging nature of MWEs in Western languages, particularly English. This progress is attributed in part to the well-established research communities and the abundant availability of computational resources. However, the same level of progress is not true for language families such as Chinese and closely related Asian languages, which continue to lag behind in this regard. While sub-word modelling has been successfully applied to many Western languages to address rare words improving phrase comprehension, and enhancing machine translation (MT) through techniques like byte-pair encoding (BPE), it cannot be applied directly to ideograph language scripts like Chinese. In this work, we conduct a systematic study of the Chinese character decomposition technology in the context of MWE-aware neural machine translation (NMT). Furthermore, we report experiments to examine how Chinese character decomposition technology contributes to the representation of the original meanings of Chinese words and characters, and how it can effectively address the challenges of translating MWEs.
Abstract (translated)
单词的意义、表示和解释在自然语言理解（NLU）、自然语言处理（NLP）和自然语言生成（NLG）任务中扮演着核心角色。这些任务中的许多固有困难源自于多词表达（MWE），它们通过引入歧义、惯用语、不常见的使用方式以及广泛的变体，使这些问题变得更加复杂。在西方语言尤其是英语中处理MWE的挑战方面已经付出了巨大的努力并取得了显著进展。这一进步部分归因于研究社区的成熟和计算资源的丰富可用性。然而，在汉语及其相关亚洲语言这样的语言家族中，并没有取得相同水平的进步；这些语言在此领域仍处于落后状态。 虽然通过应用子词模型（如字节对编码BPE技术）来解决西方语言中的罕见词汇问题，提高短语理解能力并改进机器翻译，取得了成功。但直接将这种技术应用于像汉语这样的意符文字脚本是不可行的。在这项工作中，我们系统地研究了在基于MWE感知神经机器翻译（NMT）上下文下的汉字分解技术。此外，我们报告了一些实验结果，旨在探讨汉字分解技术如何有助于中文单词和字符原始意义的表现，并且它能够有效应对翻译MWE带来的挑战。
URL
https://arxiv.org/abs/2512.15556
