Abstract
Decentralized Federated Learning (DFL) has become popular due to its robustness and avoidance of centralized coordination. In this paradigm, clients actively engage in training by exchanging models with their networked neighbors. However, DFL introduces increased costs in terms of training and communication. Existing methods focus on minimizing communication often overlooking training efficiency and data heterogeneity. To address this gap, we propose a novel \textit{sparse-to-sparser} training scheme: DA-DPFL. DA-DPFL initializes with a subset of model parameters, which progressively reduces during training via \textit{dynamic aggregation} and leads to substantial energy savings while retaining adequate information during critical learning periods. Our experiments showcase that DA-DPFL substantially outperforms DFL baselines in test accuracy, while achieving up to $5$ times reduction in energy costs. We provide a theoretical analysis of DA-DPFL's convergence by solidifying its applicability in decentralized and personalized learning. The code is available at:this https URL
Abstract (translated)
去中心化联邦学习（DFL）因其稳健性和避免集中协调而变得流行。在这种范式中，客户端通过与网络邻居交换模型来积极参与训练。然而，DFL在训练和通信方面引入了增加的成本。现有的方法通常关注最小化通信，而忽视了训练效率和数据异质性。为了填补这一空白，我们提出了一个新颖的\textit{稀疏到稀疏}训练方案：DA-DPFL。DA-DPFL以部分模型参数为基础进行初始化，在训练过程中通过动态聚合逐渐减少，从而在关键学习期间保留足够的信息。我们的实验展示了DA-DPFL在测试准确率方面明显优于DFL基线，同时实现能源成本降低至原来的5倍。我们通过固化DA-DPFL在去中心化和个性化学习上的收敛性，提供了理论分析。代码可在此处访问：https://this URL。
URL
https://arxiv.org/abs/2404.15943
