Abstract
Modern deep learning is increasingly characterized by the use of open-weight foundation models that can be fine-tuned on specialized datasets. This has led to a proliferation of expert models and adapters, often shared via platforms like HuggingFace and AdapterHub. To leverage these resources, numerous model upcycling methods have emerged, enabling the reuse of fine-tuned models in multi-task systems. A natural pipeline has thus formed to harness the benefits of transfer learning and amortize sunk training costs: models are pre-trained on general data, fine-tuned on specific tasks, and then upcycled into more general-purpose systems. A prevailing assumption is that improvements at one stage of this pipeline propagate downstream, leading to gains at subsequent steps. In this work, we challenge that assumption by examining how expert fine-tuning affects model upcycling. We show that long fine-tuning of experts that optimizes for their individual performance leads to degraded merging performance, both for fully fine-tuned and LoRA-adapted models, and to worse downstream results when LoRA adapters are upcycled into MoE layers. We trace this degradation to the memorization of a small set of difficult examples that dominate late fine-tuning steps and are subsequently forgotten during merging. Finally, we demonstrate that a task-dependent aggressive early stopping strategy can significantly improve upcycling performance.
Abstract (translated)
现代深度学习的一个显著特征是使用开放权重基础模型，这些模型可以在特定数据集上进行微调。这导致了专家模型和适配器的大量涌现，并且常常通过像HuggingFace和AdapterHub这样的平台共享。为了利用这些资源，已经出现了许多模型再利用方法，使得经过微调的模型可以被重用于多任务系统中。因此，形成了一种自然的工作流程，以实现迁移学习的好处并分摊沉没训练成本：首先是基于通用数据对模型进行预训练，然后在特定任务上对其进行微调，并最终将其重新用作更通用系统的组成部分。一种普遍的观点是，在这条管道中的每一个阶段的改进都会传递到下游环节，从而带来后续步骤上的收益。 然而，在这项工作中，我们通过研究专家级微调如何影响模型再利用来挑战这一假设。我们展示了长期对专家进行微调以优化其个体性能会导致合并时表现下降，并且当使用LoRA适配器时还会导致更差的下游结果。我们将这种退化归因于记忆少数几个困难样本的现象，这些样本在后期微调中占主导地位，但在合并阶段又被遗忘。最后，我们证明了一种依赖任务的激进早期停止策略可以显著提升再利用性能。 简而言之，过度优化特定任务会导致模型泛化能力下降，从而影响到后续的多任务或转移学习效果。我们的研究提出了一种新的微调和适配器重用策略，以提高这些模型在广泛场景中的适应性和有效性。
URL
https://arxiv.org/abs/2506.14126
