Abstract
Neural Machine Translation (NMT) models have shown remarkable performance but remain largely opaque in their decision making processes. The interpretability of these models, especially their internal attention mechanisms, is critical for building trust and verifying that these systems behave as intended. In this work, we introduce a systematic framework to quantitatively evaluate the explainability of an NMT model attention patterns by comparing them against statistical alignments and correlating them with standard machine translation quality metrics. We present a set of metrics attention entropy and alignment agreement and validate them on an English-German test subset from WMT14 using a pre trained mT5 model. Our results indicate that sharper attention distributions correlate with improved interpretability but do not always guarantee better translation quality. These findings advance our understanding of NMT explainability and guide future efforts toward building more transparent and reliable machine translation systems.
Abstract (translated)
神经机器翻译（NMT）模型展示了出色的性能，但其决策过程仍然很大程度上是不透明的。这些模型尤其是它们内部注意力机制的可解释性对于建立信任以及验证系统是否按预期运行至关重要。在这项工作中，我们引入了一个系统的框架来通过将注意力模式与统计对齐进行比较，并将其与标准机器翻译质量指标相关联，以定量评估NMT模型解释性的方法。我们提出了一组度量标准：注意熵和对齐一致性，并使用WMT14中的英语-德语测试子集以及预训练的mT5模型对其进行了验证。 我们的研究结果表明，更尖锐的注意力分布与更好的可解释性相关联，但并不总是保证更高的翻译质量。这些发现推进了我们对NMT可解释性的理解，并指导未来朝着构建更加透明和可靠的机器翻译系统而努力的方向。
URL
https://arxiv.org/abs/2412.18669
