Abstract
Robustness to label noise within data is a significant challenge in federated learning (FL). From the data-centric perspective, the data quality of distributed datasets can not be guaranteed since annotations of different clients contain complicated label noise of varying degrees, which causes the performance degradation. There have been some early attempts to tackle noisy labels in FL. However, there exists a lack of benchmark studies on comprehensively evaluating their practical performance under unified settings. To this end, we propose the first benchmark study FNBench to provide an experimental investigation which considers three diverse label noise patterns covering synthetic label noise, imperfect human-annotation errors and systematic errors. Our evaluation incorporates eighteen state-of-the-art methods over five image recognition datasets and one text classification dataset. Meanwhile, we provide observations to understand why noisy labels impair FL, and additionally exploit a representation-aware regularization method to enhance the robustness of existing methods against noisy labels based on our observations. Finally, we discuss the limitations of this work and propose three-fold future directions. To facilitate related communities, our source code is open-sourced at this https URL.
Abstract (translated)
在联邦学习（FL）中，数据标签中的鲁棒性是一个重要挑战。从数据为中心的角度来看，由于不同客户端的注释包含不同程度和复杂程度的标签噪声，因此无法保证分布式数据集的质量，这导致了性能下降。虽然已有早期尝试解决联邦学习中的噪声标签问题，但尚缺乏在统一设置下全面评估其实际表现基准研究。为此，我们提出了首个基准研究 FNBench，它考虑了三种不同的标签噪声模式（合成标签噪声、不完美的人工注释错误和系统性错误），并提供了实验调查。我们的评估涵盖了五种图像识别数据集和一种文本分类数据集上的十八种最先进的方法。 此外，我们提供观察结果来理解为什么噪声标签会损害联邦学习，并进一步利用了一种基于感知表示的正则化方法来增强现有方法对噪声标签的鲁棒性。最后，我们讨论了这项工作的局限性和提出了三个未来研究方向。为了方便相关社区，我们的源代码已开源在以下链接：[https://this-url](https://this-url)（请将此占位符替换为实际URL）。
URL
https://arxiv.org/abs/2505.06684
