Abstract
Token repetition is a typical form of multi-modal problem in fully non-autoregressive translation (NAT). In this work, we revisit the multi-modal problem in recently proposed NAT models. Our study reveals that these advanced models have introduced other types of information redundancy errors, which cannot be measured by the conventional metric - the continuous repetition ratio. By manually annotating the NAT outputs, we identify two types of information redundancy errors that correspond well to lexical and reordering multi-modality problems. Since human annotation is time-consuming and labor-intensive, we propose automatic metrics to evaluate the two types of redundant errors. Our metrics allow future studies to evaluate new methods and gain a more comprehensive understanding of their effectiveness.
Abstract (translated)
标记重复是全身非自回归翻译（NAT）中的一种典型多模态问题。在本文中，我们重新审视了最近提出的NAT模型中的多模态问题。我们的研究揭示了这些高级模型引入了其他类型的信息冗余错误，这些错误不能通过传统的指标——连续重复比来衡量。通过手动标注NAT输出，我们找出了两种与词义和排序多模态问题相符的信息冗余错误。由于人类标注费时且劳动密集，我们提出了自动指标来评估这两种冗余错误。我们的指标允许未来的研究评估这些新方法，并获得更全面的理解。
URL
https://arxiv.org/abs/2405.02673
