Abstract
This report provides a preliminary evaluation of ChatGPT for machine translation, including translation prompt, multilingual translation, and translation robustness. We adopt the prompts advised by ChatGPT to trigger its translation ability and find that the candidate prompts generally work well and show minor performance differences. By evaluating on a number of benchmark test sets, we find that ChatGPT performs competitively with commercial translation products (e.g., Google Translate) on high-resource European languages but lags behind significantly on lowresource or distant languages. As for the translation robustness, ChatGPT does not perform as well as the commercial systems on biomedical abstracts or Reddit comments but is potentially a good translator for spoken language. Scripts and data: this https URL
Abstract (translated)
本报告对 ChatGPT 进行机器翻译的初步评估,包括翻译提示、多语言翻译和翻译稳定性。我们采用 ChatGPT 建议的提示来触发其翻译能力,发现候选人提示 generally 工作良好并表现出微小的性能差异。通过评估多个基准测试集,我们发现 ChatGPT 在资源充足的欧洲语言方面表现 competitive ly with 商业翻译产品 (如 Google Translate) 但对于资源较少或距离较远的语言方面表现不佳。关于翻译稳定性, ChatGPT 在生物医学摘要或 Reddit 评论方面的表现不如商业系统,但可能对于口语语言方面表现良好。脚本和数据:此 https URL。
URL
https://arxiv.org/abs/2301.08745
