Abstract
Recent work in cross-language information retrieval (CLIR), where queries and documents are in different languages, has shown the benefit of the Translate-Distill framework that trains a cross-language neural dual-encoder model using translation and distillation. However, Translate-Distill only supports a single document language. Multilingual information retrieval (MLIR), which ranks a multilingual document collection, is harder to train than CLIR because the model must assign comparable relevance scores to documents in different languages. This work extends Translate-Distill and propose Multilingual Translate-Distill (MTD) for MLIR. We show that ColBERT-X models trained with MTD outperform their counterparts trained ith Multilingual Translate-Train, which is the previous state-of-the-art training approach, by 5% to 25% in nDCG@20 and 15% to 45% in MAP. We also show that the model is robust to the way languages are mixed in training batches. Our implementation is available on GitHub.
Abstract (translated)
近年来，在跨语言信息检索（CLIR）领域，使用不同语言的查询和文档进行研究，已经证明了使用翻译和蒸馏训练跨语言神经双编码器模型的优势。然而，Translate-Distill 仅支持单个文档语言。多语言信息检索（MLIR）比 CLIR 更难训练，因为模型必须为不同语言的文档分配相似的 relevance 分数。这项工作扩展了 Translate-Distill，并提出了多语言 Translate-Distill（MTD）用于 MLIR。我们证明了，使用 ColBERT-X 模型在 MTD 训练的跨语言预训练模型在 nDCG@20 和 MAP 上优于使用多语言 Translate-Train 的先前最先进状态，其性能分别提高了 5% 到 25% 和 15% 到 45%。我们还证明了模型对训练批次中语言混合的这种方式非常鲁棒。我们的实现可以在 GitHub 上获取。
URL
https://arxiv.org/abs/2405.00977
