Abstract
We introduce V-Agent, a novel multi-agent platform designed for advanced video search and interactive user-system conversations. By fine-tuning a vision-language model (VLM) with a small video preference dataset and enhancing it with a retrieval vector from an image-text retrieval model, we overcome the limitations of traditional text-based retrieval systems in multimodal scenarios. The VLM-based retrieval model independently embeds video frames and audio transcriptions from an automatic speech recognition (ASR) module into a shared multimodal representation space, enabling V-Agent to interpret both visual and spoken content for context-aware video search. This system consists of three agents-a routing agent, a search agent, and a chat agent-that work collaboratively to address user intents by refining search outputs and communicating with users. The search agent utilizes the VLM-based retrieval model together with an additional re-ranking module to further enhance video retrieval quality. Our proposed framework demonstrates state-of-the-art zero-shot performance on the MultiVENT 2.0 benchmark, highlighting its potential for both academic research and real-world applications.
Abstract (translated)
我们介绍V-Agent，这是一个新颖的多代理平台，专为高级视频搜索和用户系统交互对话而设计。通过使用一个小规模的视频偏好数据集对视觉-语言模型（VLM）进行微调，并结合从图像文本检索模型中提取的检索向量，我们克服了传统基于文本的检索系统在跨模态场景中的局限性。该VLM基的检索模型独立地将视频帧和自动语音识别（ASR）模块生成的文字转录嵌入到一个共享的多模式表示空间中，使V-Agent能够理解视觉内容和口语表达，从而实现情境感知的视频搜索。这个系统包括三个代理——路由代理、搜索代理和聊天代理，它们协作工作以满足用户的需求，通过优化检索输出并与用户沟通来解决问题。搜索代理利用基于VLM的检索模型，并结合额外的重排序模块进一步提高视频检索质量。我们提出的框架在MultiVENT 2.0基准测试中展示了最先进的零样本性能，突显了其在学术研究和实际应用中的潜力。
URL
https://arxiv.org/abs/2512.16925
