Abstract
While transfer learning is an advantageous strategy, it overlooks the opportunity to leverage knowledge from numerous available models online. Addressing this multi-source transfer learning problem is a promising path to boost adaptability and cut re-training costs. However, existing approaches are inherently coarse-grained, lacking the necessary precision for granular knowledge extraction and the aggregation efficiency required to fuse knowledge from either a large number of source models or those with high parameter counts. We address these limitations by leveraging Singular Value Decomposition (SVD) to first decompose each source model into its elementary, rank-one components. A subsequent aggregation stage then selects only the most salient components from all sources, thereby overcoming the previous efficiency and precision limitations. To best preserve and leverage the synthesized knowledge base, our method adapts to the target task by fine-tuning only the principal singular values of the merged matrix. In essence, this process only recalibrates the importance of top SVD components. The proposed framework allows for efficient transfer learning, is robust to perturbations both at the input level and in the parameter space (e.g., noisy or pruned sources), and scales well computationally.
Abstract (translated)
虽然迁移学习是一种有利的策略，但它忽略了利用在线众多可用模型知识的机会。解决多源迁移学习问题是一条提高适应性和削减再训练成本的有希望的道路。然而，现有的方法本质上是粗粒度的，缺乏从大量源模型或参数数量高的模型中提取精细知识和融合知识所需的精度与效率。我们通过利用奇异值分解（SVD）来克服这些限制：首先将每个源模型分解为其基本的一级分量。随后的聚合阶段则仅选择所有来源中最显著的部分，从而解决以前的方法在效率和精度上的局限性。为了最大程度地保存并有效利用合成的知识库，我们的方法通过微调合并矩阵的主要奇异值来适应目标任务，这一过程仅重新校准顶级SVD成分的重要性。所提出的框架能够实现高效的迁移学习，并且对输入层面和参数空间中的扰动（例如嘈杂或剪枝后的源模型）具有鲁棒性，同时在计算上也表现良好。
URL
https://arxiv.org/abs/2508.19353
