Abstract
Large-scale Pretrained Language Models~(LLMs), such as ChatGPT and GPT4, have shown strong abilities in multilingual translations, without being explicitly trained on parallel corpora. It is interesting how the LLMs obtain their ability to carry out translation instructions for different languages. In this paper, we present a detailed analysis by finetuning a multilingual pretrained language model, XGLM-7B, to perform multilingual translation following given instructions. Firstly, we show that the multilingual LLMs have stronger translation abilities than previously demonstrated. For a certain language pair, the performance depends on both the language families and the amount of data used in the pretraining phase. Secondly, we find that LLMs' ability to carry out translation instructions relies on the understanding of translation instruction and the alignment among different languages. With proper enhancement, LLMs could perform the translation task well even for those language pairs unseen during the instruction tuning phase.
Abstract (translated)
大型预训练语言模型(LLMs)，如ChatGPT和GPT4，在多语言翻译方面表现出强大的能力，而并没有在并行语料库上 explicitly 训练。这很有趣，LLMs 如何获得对不同语言的翻译指令的能力。在本文中，我们将通过微调一个多语言预训练语言模型XGLM-7B，按照给定指令进行多语言翻译，详细分析。首先，我们表明，多语言LLMs比先前表现出的翻译能力更强。对于某些语言对，性能取决于语言家族和使用预训练阶段的数据量。其次，我们发现LLMs的翻译指令执行能力取决于对翻译指令的理解和不同语言的对齐。通过适当的增强，LLMs甚至可以在指令调整阶段未访问的语言对上表现良好。
URL
https://arxiv.org/abs/2305.15083
