Abstract
Federated learning (FL) enables decentralized clients to train a model collaboratively without sharing local data. A key distinction between FL and centralized learning is that clients' data are non-independent and identically distributed, which poses significant challenges in training a global model that generalizes well across heterogeneous local data distributions. In this paper, we analyze the convergence of overparameterized FedAvg with gradient descent (GD). We prove that the impact of data heterogeneity diminishes as the width of neural networks increases, ultimately vanishing when the width approaches infinity. In the infinite-width regime, we further prove that both the global and local models in FedAvg behave as linear models, and that FedAvg achieves the same generalization performance as centralized learning with the same number of GD iterations. Extensive experiments validate our theoretical findings across various network architectures, loss functions, and optimization methods.
Abstract (translated)
联邦学习（FL）允许分散的客户端在不共享本地数据的情况下协作训练模型。联邦学习与集中式学习之间的一个关键区别在于，客户端的数据是非独立且异构分布的，这给跨不同局部数据分布泛化的全局模型训练带来了重大挑战。 本文分析了使用梯度下降法（GD）进行超参数化FedAvg的收敛性。我们证明随着神经网络宽度的增加，数据异质性的负面影响逐渐减少，并在宽度趋于无穷大时完全消失。在网络宽度无限的情况下，我们进一步证明了FedAvg中的全局模型和局部模型都表现得像线性模型一样，并且当迭代次数相同时，FedAvg达到了与集中式学习相同的泛化性能。 广泛的实验验证了我们的理论发现，在各种网络架构、损失函数以及优化方法下均适用。
URL
https://arxiv.org/abs/2508.12576
