Abstract
MTL is a learning paradigm that effectively leverages both task-specific and shared information to address multiple related tasks simultaneously. In contrast to STL, MTL offers a suite of benefits that enhance both the training process and the inference efficiency. MTL's key advantages encompass streamlined model architecture, performance enhancement, and cross-domain generalizability. Over the past twenty years, MTL has become widely recognized as a flexible and effective approach in various fields, including CV, NLP, recommendation systems, disease prognosis and diagnosis, and robotics. This survey provides a comprehensive overview of the evolution of MTL, encompassing the technical aspects of cutting-edge methods from traditional approaches to deep learning and the latest trend of pretrained foundation models. Our survey methodically categorizes MTL techniques into five key areas: regularization, relationship learning, feature propagation, optimization, and pre-training. This categorization not only chronologically outlines the development of MTL but also dives into various specialized strategies within each category. Furthermore, the survey reveals how the MTL evolves from handling a fixed set of tasks to embracing a more flexible approach free from task or modality constraints. It explores the concepts of task-promptable and -agnostic training, along with the capacity for ZSL, which unleashes the untapped potential of this historically coveted learning paradigm. Overall, we hope this survey provides the research community with a comprehensive overview of the advancements in MTL from its inception in 1997 to the present in 2023. We address present challenges and look ahead to future possibilities, shedding light on the opportunities and potential avenues for MTL research in a broad manner. This project is publicly available at this https URL.
Abstract (translated)
MTL是一种学习范式，有效利用任务特定和共享信息来同时解决多个相关任务。与STL相比，MTL提供了一系列增强训练过程和推理效率的优势。MTL的关键优势包括简化模型架构、性能提升和跨领域泛化。在过去的二十年中，MTL已经成为许多领域广泛认可的灵活有效的解决方案，包括CV、自然语言处理、推荐系统、疾病预后和诊断、以及机器人领域。本次调查全面回顾了MTL的发展历程，从传统方法的尖端技术到深度学习的最新趋势，以及预训练基础模型的最新趋势。我们的调查系统地分类MTL技术为五个关键领域：正则化、关系学习、特征传播、优化和预训练。这种分类不仅按时间顺序描述了MTL的发展，还深入研究了每个领域的各种专业策略。此外，调查揭示了MTL如何从处理固定任务转变为更加灵活的方法，摆脱了任务或模型约束。它探讨了任务提示的和无条件的训练概念，以及ZSL（零样本学习）的能力，揭示了这一历史悠久的值得称赞的学习范式所蕴含的潜力。总的来说，我们希望通过这次调查为研究社区提供MTL从1997年创立到2023年的全面概述。我们关注当前的挑战，展望未来的机遇，以一种全面的方式揭示MTL研究在各个领域的机会和潜在途径。这个项目在https://这个URL上公开可用。
URL
https://arxiv.org/abs/2404.18961
