Abstract
Recent advances in neural methods have led to substantial improvement in the quality of Neural Machine Translation (NMT) systems. However, these systems frequently produce translations with inaccurate gender (Stanovsky et al., 2019), which can be traced to bias in training data. Saunders and Byrne (2020) tackle this problem with a handcrafted dataset containing balanced gendered profession words. By using this data to fine-tune an existing NMT model, they show that gender bias can be significantly mitigated, albeit at the expense of translation quality due to catastrophic forgetting. They recover some of the lost quality with modified training objectives or additional models at inference. We find, however, that simply supplementing the handcrafted dataset with a random sample from the base model training corpus is enough to significantly reduce the catastrophic forgetting. We also propose a novel domain-adaptation technique that leverages in-domain data created with the counterfactual data generation techniques proposed by Zmigrod et al. (2019) to further improve accuracy on the WinoMT challenge test set without significant loss in translation quality. We show its effectiveness in NMT systems from English into three morphologically rich languages French, Spanish, and Italian. The relevant dataset and code will be available at Github.
Abstract (translated)
近年来，在神经方法上的进展已经使得神经机器翻译（NMT）系统的质量得到了显著提高。然而，这些系统经常产生带有不准确性别（Stanovsky等人，2019）的翻译，这种不准确可能源于训练数据中的偏见。Saunders和Byrne（2020）通过包含平衡性别职业词汇的手动数据集来解决这个问题的方法。通过使用这个数据来微调现有的 NMT 模型，他们表明，虽然性别偏见可以明显减轻，但翻译质量可能会因为灾难性忘记而下降。他们通过修改训练目标或添加推理过程中的额外模型来恢复一些丢失的翻译质量。然而，我们发现，仅通过在手工数据集上补充来自基础模型训练语料库的随机样本，就可以显著地降低灾难性忘记。我们还提出了一种新的领域自适应技术，它利用了Zmigrod等人（2019）提出的反事实数据生成技术来进一步改善 WinoMT 挑战测试集中的准确性，而不会导致翻译质量的显著下降。我们在英语、法语、意大利等三种语义丰富的语言的 NMT 系统中证明了它的有效性。有关数据集和代码将在 Github 上发布。
URL
https://arxiv.org/abs/2311.16362
