Abstract
The increasing diversity and scale of video data demand retrieval systems capable of multimodal understanding, adaptive reasoning, and domain-specific knowledge integration. This paper presents LLandMark, a modular multi-agent framework for landmark-aware multimodal video retrieval to handle real-world complex queries. The framework features specialized agents that collaborate across four stages: query parsing and planning, landmark reasoning, multimodal retrieval, and reranked answer synthesis. A key component, the Landmark Knowledge Agent, detects cultural or spatial landmarks and reformulates them into descriptive visual prompts, enhancing CLIP-based semantic matching for Vietnamese scenes. To expand capabilities, we introduce an LLM-assisted image-to-image pipeline, where a large language model (Gemini 2.5 Flash) autonomously detects landmarks, generates image search queries, retrieves representative images, and performs CLIP-based visual similarity matching, removing the need for manual image input. In addition, an OCR refinement module leveraging Gemini and LlamaIndex improves Vietnamese text recognition. Experimental results show that LLandMark achieves adaptive, culturally grounded, and explainable retrieval performance.
Abstract (translated)
视频数据的多样性和规模日益增大，这要求检索系统具备多模态理解、自适应推理和领域特定知识集成的能力。本文提出了LLandMark框架，这是一个模块化的多代理框架，用于地标感知的多模态视频检索，以处理现实世界中的复杂查询。该框架包括四个阶段的专业代理：查询解析与规划、地标推理、多模态检索以及重新排序答案合成。 其中一个重要组件是地标知识代理，它可以检测文化或空间地标，并将其转化为描述性的视觉提示，从而增强CLIP（对比语言-图像预训练模型）在越南场景中的语义匹配能力。为了进一步扩展功能，我们引入了一个由大型语言模型（Gemini 2.5 Flash）支持的图片到图片处理管道，在这个过程中，大型语言模型自主地检测地标、生成图片搜索查询、检索代表性图片，并进行基于CLIP的视觉相似性匹配，从而消除了对人工输入图片的需求。此外，一个利用Gemini和LlamaIndex优化越南文本识别能力的文字光学字符识别（OCR）模块也被引入进来。 实验结果显示，LLandMark框架能够实现适应性强、文化根基深厚且具有可解释性的检索性能。
URL
https://arxiv.org/abs/2603.02888
