Paper Reading AI Learner

FedFA: Federated Feature Augmentation

2023-01-30 15:39:55
Tianfei Zhou, Ender Konukoglu

Abstract

Federated learning is a distributed paradigm that allows multiple parties to collaboratively train deep models without exchanging the raw data. However, the data distribution among clients is naturally non-i.i.d., which leads to severe degradation of the learnt model. The primary goal of this paper is to develop a robust federated learning algorithm to address feature shift in clients' samples, which can be caused by various factors, e.g., acquisition differences in medical imaging. To reach this goal, we propose FedFA to tackle federated learning from a distinct perspective of federated feature augmentation. FedFA is based on a major insight that each client's data distribution can be characterized by statistics (i.e., mean and standard deviation) of latent features; and it is likely to manipulate these local statistics globally, i.e., based on information in the entire federation, to let clients have a better sense of the underlying distribution and therefore alleviate local data bias. Based on this insight, we propose to augment each local feature statistic probabilistically based on a normal distribution, whose mean is the original statistic and variance quantifies the augmentation scope. Key to our approach is the determination of a meaningful Gaussian variance, which is accomplished by taking into account not only biased data of each individual client, but also underlying feature statistics characterized by all participating clients. We offer both theoretical and empirical justifications to verify the effectiveness of FedFA. Our code is available at this https URL.

Abstract (translated)

分布式学习是一种分布式范式,允许多个主体协作训练深度模型,而无需交换原始数据。然而,在客户之间的数据分布中,自然不是i.i.d.,这导致学习模型严重退化。本文的主要目标是开发一种可靠的分布式学习算法,解决客户样本中特征的变化,这种情况可能是由于各种因素,例如医学影像采集的差异。为了实现这一目标,我们提出了FedFA,以从分布式特征增强的独特视角来处理分布式学习。FedFA基于一个重要的洞察力,即每个客户的数据分布可以用隐特征的统计学特征来描述;它可能在全球范围内操纵这些本地统计量,即基于整个联邦的信息,使客户更好地理解底层分布,从而减轻本地数据偏差。基于这一洞察力,我们提出了基于概率分布的每个本地特征统计增强,其均值是原始统计量,而变异度衡量增强范围。我们的方法的关键点是确定有意义的高斯变异度,这是通过考虑到每个参与客户的数据偏差以及所有客户描述的隐特征统计学特征来实现的。我们提供了理论性和实证性证明,以验证FedFA的有效性。我们的代码在此httpsURL上可用。

URL

https://arxiv.org/abs/2301.12995

PDF

https://arxiv.org/pdf/2301.12995.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot