Abstract
Federated learning (FL) is a promising approach for solving multilingual tasks, potentially enabling clients with their own language-specific data to collaboratively construct a high-quality neural machine translation (NMT) model. However, communication constraints in practical network systems present challenges for exchanging large-scale NMT engines between FL parties. In this paper, we propose a meta-learning-based adaptive parameter selection methodology, MetaSend, that improves the communication efficiency of model transmissions from clients during FL-based multilingual NMT training. Our approach learns a dynamic threshold for filtering parameters prior to transmission without compromising the NMT model quality, based on the tensor deviations of clients between different FL rounds. Through experiments on two NMT datasets with different language distributions, we demonstrate that MetaSend obtains substantial improvements over baselines in translation quality in the presence of a limited communication budget.
Abstract (translated)
联邦学习（FL）是一种解决多语言任务的的有前途的方法，可能使具有自己语言特定数据的所有客户端协同构建高质量的语言机器翻译（NMT）模型。然而，实际网络系统中的通信限制为在FL各方之间交换大规模NMT引擎设置了挑战。在本文中，我们提出了一个基于元学习的自适应参数选择方法，元发送，以提高基于FL的多语言NMT训练中客户端模型的传输效率。我们的方法基于客户端之间不同FL轮的张量偏差来学习动态阈值，在传输参数之前不牺牲NMT模型质量。通过在两个具有不同语言分布的NMT数据集上的实验，我们证明了元发送在有限通信预算下显著提高了翻译质量。
URL
https://arxiv.org/abs/2401.07456
