Abstract
Foundation Models (FMs) have demonstrated strong generalization across diverse vision tasks. However, their deployment in federated settings is hindered by high computational demands, substantial communication overhead, and significant inference costs. We propose DSFedMed, a dual-scale federated framework that enables mutual knowledge distillation between a centralized foundation model and lightweight client models for medical image segmentation. To support knowledge distillation, a set of high-quality medical images is generated to replace real public datasets, and a learnability-guided sample selection strategy is proposed to enhance efficiency and effectiveness in dual-scale distillation. This mutual distillation enables the foundation model to transfer general knowledge to lightweight clients, while also incorporating client-specific insights to refine the foundation model. Evaluations on five medical imaging segmentation datasets show that DSFedMed achieves an average 2 percent improvement in Dice score while reducing communication costs and inference time by nearly 90 percent compared to existing federated foundation model baselines. These results demonstrate significant efficiency gains and scalability for resource-limited federated deployments.
Abstract (translated)
基础模型（FMs）在各种视觉任务中展现出了强大的泛化能力。然而，它们在联邦环境中的部署由于计算需求高、通信开销大以及推理成本显著而受到阻碍。为此，我们提出了DSFedMed，这是一种双尺度联邦框架，该框架允许集中式基础模型与轻量级客户端模型之间进行相互知识蒸馏，以用于医学图像分割任务中。为了支持这种知识蒸馏过程，生成了一组高质量的医学图像来替代真实的公开数据集，并提出了一种基于可学习性引导的样本选择策略，以提高双尺度蒸馏中的效率和效果。该双向蒸馏方法使得基础模型能够将通用知识传递给轻量级客户端，同时也能吸收来自客户端的具体见解以优化自身。 在五个医学影像分割数据集上的评估表明，DSFedMed相较于现有的联邦基础模型基线方案，在Dice分数上平均提高了2%，并且减少了近90%的通信成本和推理时间。这些结果展示了资源受限环境下联邦部署的有效性提升与可扩展性的显著进步。
URL
https://arxiv.org/abs/2601.16073
