Abstract
Instruction tuning of language models has demonstrated the ability to enhance model generalization to unseen tasks via in-context learning using a few examples. However, typical supervised learning still requires a plethora of downstream training data for finetuning. Often in real-world situations, there is a scarcity of data available for finetuning, falling somewhere between few shot inference and fully supervised finetuning. In this work, we demonstrate the sample efficiency of instruction tuned models over various tasks by estimating the minimal downstream training data required by them to perform transfer learning and match the performance of state-of-the-art (SOTA) supervised models. We conduct experiments on 119 tasks from Super Natural Instructions (SuperNI) in both the single task learning (STL) and multi task learning (MTL) settings. Our findings reveal that, in the STL setting, instruction tuned models equipped with 25% of the downstream train data surpass the SOTA performance on the downstream tasks. In the MTL setting, an instruction tuned model trained on only 6% of downstream training data achieve SOTA, while using 100% of the training data results in a 3.69% points improvement (ROUGE-L 74.68) over the previous SOTA. We conduct an analysis on T5 vs Tk-Instruct by developing several baselines to demonstrate that instruction tuning aids in increasing both sample efficiency and transfer learning. Additionally, we observe a consistent ~4% performance increase in both settings when pre-finetuning is performed with instructions. Finally, we conduct a categorical study and find that contrary to previous results, tasks in the question rewriting and title generation categories suffer from instruction tuning.
Abstract (translated)
指令优化语言模型已经证明了通过使用几个例子来提高模型对未完成任务的泛化能力的能力。然而,典型的监督学习仍然需要大量的后续训练数据来进行微调。通常,在现实世界的情况下,微调数据的资源非常有限,处于几个Shot推断和完全监督微调之间的中间位置。在本文中,我们使用Super Natural Instructions(超级指令)中的119个任务进行了实验,同时在单任务学习和多任务学习环境中进行了测试。我们的发现表明,在STL环境中,指令优化模型所拥有的25%的后续训练数据超过了后续任务的性能表现(与SOTA相比)。在MTL环境中,只使用后续训练数据训练的指令优化模型达到了SOTA表现,而使用全部训练数据则导致了3.69%点的进步(ROUGE-L 74.68)高于之前的SOTA表现。我们对T5和Tk-Instruct进行了分析,以开发多个基准来表明指令优化有助于增加样本效率和迁移学习。此外,我们在两个环境中观察到一致的 ~4%的性能提升,在执行预微调指令之前。最后,我们进行了分类研究,并发现与之前的结果相反,问题改写和标题生成任务中的任务受到指令优化的影响。
URL
https://arxiv.org/abs/2306.05539
