Abstract
Many-to-one neural machine translation systems improve over one-to-one systems when training data is scarce. In this paper, we design and test a novel algorithm for selecting the language of minibatches when training such systems. The algorithm changes the language of the minibatch when the weights of the model do not evolve significantly, as measured by the smoothed KL divergence between all layers of the Transformer network. This algorithm outperforms the use of alternating monolingual batches, but not the use of shuffled batches, in terms of translation quality (measured with BLEU and COMET) and convergence speed.
Abstract (translated)
许多对一神经机器翻译系统在训练数据有限的情况下表现优于一对一系统。在本文中，我们设计了一种新算法来选择在训练过程中语言，该算法在模型权重在平滑的KL散度定义的Transformer网络的所有层之间时没有显著变化时更改minibatch的语言。与交替单语料库相比，该算法在翻译质量（使用BLEU和COMET）和收敛速度方面都表现出色。而与洗牌单语料库相比，该算法的表现并不突出。
URL
https://arxiv.org/abs/2410.04147
