Abstract
Federated learning aims to tackle the ``isolated data island" problem, where it trains a collective model from physically isolated clients while safeguarding the privacy of users' data. However, supervised federated learning necessitates that each client labels their data for training, which can be both time-consuming and resource-intensive, and may even be impractical for edge devices. Moreover, the training and transmission of deep models present challenges to the computation and communication capabilities of the clients. To address these two inherent challenges in supervised federated learning, we propose a novel lightweight unsupervised federated learning approach that leverages unlabeled data on each client to perform lightweight model training and communication by harnessing pretrained vision-language models, such as CLIP. By capitalizing on the zero-shot prediction capability and the well-trained image encoder of the pre-trained CLIP model, we have carefully crafted an efficient and resilient self-training approach. This method refines the initial zero-shot predicted pseudo-labels of unlabeled instances through the sole training of a linear classifier on top of the fixed image encoder. Additionally, to address data heterogeneity within each client, we propose a class-balanced text feature sampling strategy for generating synthetic instances in the feature space to support local training. Experiments are conducted on multiple benchmark datasets. The experimental results demonstrate that our proposed method greatly enhances model performance in comparison to CLIP's zero-shot predictions and even outperforms supervised federated learning benchmark methods given limited computational and communication overhead.
Abstract (translated)
联邦学习旨在解决“孤立数据岛”问题，即在保护用户数据隐私的前提下，从物理上隔离的客户端训练一个集体模型。然而，监督式联邦学习需要每个客户端为训练数据进行标注，这可能耗时且资源密集，甚至对于边缘设备来说可能不可行。此外，深度模型的训练和传输对客户端的计算和通信能力提出了挑战。为了应对监督式联邦学习中的这两个固有挑战，我们提出了一种新颖的轻量级无监督联邦学习方法，它通过利用预训练的视觉-语言模型（如CLIP）在每个客户端上进行轻量级模型训练和通信，从而实现了 harnessing the zero-shot prediction capability and the well-trained image encoder of the pre-trained CLIP model. 通过充分利用预训练CLIP模型的零 shot预测能力和预训练的图像编码器良好的图像处理能力，我们精心设计了一种高效且鲁棒的自我训练方法。通过在固定图像编码器之上仅通过线性分类器进行训练，我们通过自我训练对未标注实例的初始零 shot预测伪标签进行了优化。此外，为了解决每个客户端内数据异质性问题，我们提出了一个类平衡文本特征抽样策略，在特征空间中生成合成实例以支持局部训练。我们在多个基准数据集上进行了实验。实验结果表明，与CLIP的零 shot预测相比，我们提出的方法在模型性能方面 greatly增强了效果，甚至在有限计算和通信开销下，甚至超过了监督式联邦学习基准方法。
URL
https://arxiv.org/abs/2404.11046
