Abstract
Recently, we have witnessed the great success of the generalist model in natural language processing. The generalist model is a general framework trained with massive data and is able to process various downstream tasks simultaneously. Encouraged by their impressive performance, an increasing number of researchers are venturing into the realm of applying these models to computer vision tasks. However, the inputs and outputs of vision tasks are more diverse, and it is difficult to summarize them as a unified representation. In this paper, we provide a comprehensive overview of the vision generalist models, delving into their characteristics and capabilities within the field. First, we review the background, including the datasets, tasks, and benchmarks. Then, we dig into the design of frameworks that have been proposed in existing research, while also introducing the techniques employed to enhance their performance. To better help the researchers comprehend the area, we take a brief excursion into related domains, shedding light on their interconnections and potential synergies. To conclude, we provide some real-world application scenarios, undertake a thorough examination of the persistent challenges, and offer insights into possible directions for future research endeavors.
Abstract (translated)
最近，我们在自然语言处理领域见证了通用模型的巨大成功。通用模型是在海量数据上训练的一个框架，能够同时处理多种下游任务。受到这些模型出色性能的鼓舞，越来越多的研究人员开始尝试将其应用于计算机视觉任务中。然而，视觉任务的输入和输出更加多样，难以用统一的形式表示它们。在本文中，我们全面概述了视觉通用模型，并深入探讨了其特性和能力。首先，我们将回顾背景知识，包括数据集、任务及基准测试。然后，我们将讨论现有研究中提出的框架设计方法，同时介绍用于提升性能的技术手段。为了更好地帮助研究人员理解这一领域，我们将简要介绍相关的其他领域，揭示它们之间的相互联系和潜在协同作用。最后，我们提供了一些实际应用案例，并对持久存在的挑战进行了深入分析，同时也提供了对未来研究方向的见解。
URL
https://arxiv.org/abs/2506.09954
