Abstract
Recently, many studies have shown the efficiency of using Bidirectional Encoder Representations from Transformers (BERT) in various Natural Language Processing (NLP) tasks. Specifically, English spelling correction task that uses Encoder-Decoder architecture and takes advantage of BERT has achieved state-of-the-art result. However, to our knowledge, there is no implementation in Vietnamese yet. Therefore, in this study, a combination of Transformer architecture (state-of-the-art for Encoder-Decoder model) and BERT was proposed to deal with Vietnamese spelling correction. The experiment results have shown that our model outperforms other approaches as well as the Google Docs Spell Checking tool, achieves an 86.24 BLEU score on this task.
Abstract (translated)
最近，许多研究都表明了使用双向编码器表示来自Transformer（BERT）在各种自然语言处理（NLP）任务中的效率。具体来说，利用BERT的编码器-解码器架构的英语拼写纠错任务已经达到了最先进的水平。然而，据我们所知，在越南还没有实现。因此，在本文中，我们提出了结合Transformer架构（对于编码器-解码器模型状态最佳）和BERT来处理越南拼写纠错的想法。实验结果表明，我们的模型在表现为其他方法和Google Docs拼写检查工具方面均表现优异，并且达到了86.24 BLEU得分。
URL
https://arxiv.org/abs/2405.02573
