Abstract
Recently, vision-language pretraining has emerged as a transformative technique that integrates the strengths of both visual and textual modalities, resulting in powerful vision-language models (VLMs). Leveraging web-scale pretraining data, these models exhibit strong zero-shot capabilities. However, their performance often deteriorates when confronted with domain-specific or specialized generalization tasks. To address this, a growing body of research focuses on transferring or generalizing the rich knowledge embedded in VLMs to various downstream applications. This survey aims to comprehensively summarize the generalization settings, methodologies, benchmarking and results in VLM literatures. Delving into the typical VLM structures, current literatures are categorized into prompt-based, parameter-based and feature-based methods according to the transferred modules. The differences and characteristics in each category are furthered summarized and discussed by revisiting the typical transfer learning (TL) settings, providing novel interpretations for TL in the era of VLMs. Popular benchmarks for VLM generalization are further introduced with thorough performance comparisons among the reviewed methods. Following the advances in large-scale generalizable pretraining, this survey also discusses the relations and differences between VLMs and up-to-date multimodal large language models (MLLM), e.g., DeepSeek-VL. By systematically reviewing the surging literatures in vision-language research from a novel and practical generalization prospective, this survey contributes to a clear landscape of current and future multimodal researches.
Abstract (translated)
最近，视觉-语言预训练作为一种变革性技术应运而生，它结合了视觉和文本模态的优势，产生了强大的视觉-语言模型（VLM）。通过利用大规模的网络预训练数据，这些模型展示了强大的零样本学习能力。然而，在面对特定领域或专业化的泛化任务时，它们的表现往往会下降。为了解决这一问题，越来越多的研究专注于将嵌入在VLM中的丰富知识转移到各种下游应用中。本综述旨在全面总结视觉-语言文献中的泛化设置、方法论、基准测试和结果。深入探讨了典型的VLM结构，并根据转移模块的不同，当前的文献被分类为基于提示（prompt-based）、基于参数（parameter-based）和基于特征（feature-based）的方法。通过回顾典型的迁移学习（TL）设定，对每一类别的差异和特点进行了进一步总结与讨论，提供了在VLM时代TL的新颖解释。本综述还介绍了用于VLM泛化的流行基准，并对比了所审查方法的性能表现。随着大规模可泛化预训练的进步，该综述还讨论了VLM与最新多模态大型语言模型（MLLM）如DeepSeek-VL之间的关系和差异。通过从新颖且实用的角度系统地回顾视觉-语言研究中的激增文献，本综述有助于清晰描绘当前和未来的多模态研究图景。
URL
https://arxiv.org/abs/2506.18504
