Abstract
Robustness to malicious attacks is crucial for practical decentralized signal processing and machine learning systems. A typical example of such attacks is label poisoning, meaning that some agents possess corrupted local labels and share models trained on these poisoned data. To defend against malicious attacks, existing works often focus on designing robust aggregators; meanwhile, the weighted mean aggregator is typically considered a simple, vulnerable baseline. This paper analyzes the robustness of decentralized gradient descent under label poisoning attacks, considering both robust and weighted mean aggregators. Theoretical results reveal that the learning errors of robust aggregators depend on the network topology, whereas the performance of weighted mean aggregator is topology-independent. Remarkably, the weighted mean aggregator, although often considered vulnerable, can outperform robust aggregators under sufficient heterogeneity, particularly when: (i) the global contamination rate (i.e., the fraction of poisoned agents for the entire network) is smaller than the local contamination rate (i.e., the maximal fraction of poisoned neighbors for the regular agents); (ii) the network of regular agents is disconnected; or (iii) the network of regular agents is sparse and the local contamination rate is high. Empirical results support our theoretical findings, highlighting the important role of network topology in the robustness to label poisoning attacks.
Abstract (translated)
对抗恶意攻击的能力对于实际的分布式信号处理和机器学习系统至关重要。这类攻击的一个典型例子是标签投毒，即某些代理拥有被篡改的本地标签，并且会分享在其投毒数据上训练出来的模型。为了抵御恶意攻击，现有的研究工作往往集中在设计鲁棒聚合器；与此同时，加权平均聚合器通常被视为一种简单而脆弱的基础方法。 本文分析了在标签投毒攻击下的分布式梯度下降算法的稳健性，同时考虑了鲁棒和加权均值聚合器的效果。理论结果表明，鲁棒聚合器的学习误差依赖于网络拓扑结构，而加权均值聚合器的表现则与拓扑无关。值得注意的是，尽管加权平均聚合器通常被认为脆弱，但它在足够异质的情况下可以优于鲁棒聚合器，特别是在以下情况： (i) 全局投毒率（即整个网络中投毒代理的比例）低于局部投毒率（即常规代理的最大投毒邻居比例）； (ii) 常规代理的网络是断开连接的；或者 (iii) 常规代理的网络很稀疏且局部投毒率很高。 实验证据支持了我们的理论发现，突显了网络拓扑结构在抵御标签投毒攻击中的重要作用。
URL
https://arxiv.org/abs/2601.02682
