Abstract
Retrieval-Augmented Generation (RAG) mitigates hallucinations in Multimodal Large Language Models (MLLMs), yet existing systems struggle with complex cross-modal reasoning. Flat vector retrieval often ignores structural dependencies, while current graph-based methods rely on costly ``translation-to-text'' pipelines that discard fine-grained visual information. To address these limitations, we propose \textbf{MG$^2$-RAG}, a lightweight \textbf{M}ulti-\textbf{G}ranularity \textbf{G}raph \textbf{RAG} framework that jointly improves graph construction, modality fusion, and cross-modal retrieval. MG$^2$-RAG constructs a hierarchical multimodal knowledge graph by combining lightweight textual parsing with entity-driven visual grounding, enabling textual entities and visual regions to be fused into unified multimodal nodes that preserve atomic evidence. Building on this representation, we introduce a multi-granularity graph retrieval mechanism that aggregates dense similarities and propagates relevance across the graph to support structured multi-hop reasoning. Extensive experiments across four representative multimodal tasks (i.e., retrieval, knowledge-based VQA, reasoning, and classification) demonstrate that MG$^2$-RAG consistently achieves state-of-the-art performance while reducing graph construction overhead with an average 43.3$\times$ speedup and 23.9$\times$ cost reduction compared with advanced graph-based frameworks.
Abstract (translated)
检索增强生成（RAG）可缓解多模态大语言模型（MLLMs）中的幻觉问题，但现有系统在复杂跨模态推理方面仍存在不足。扁平向量检索常忽略结构依赖关系，而当前基于图的方法依赖成本高昂的“文本转换流水线”，会丢失细粒度的视觉信息。为应对这些局限，我们提出了\textbf{MG$^2$-RAG}，一个轻量级的\textbf{多粒度图RAG}框架，可协同优化图构建、模态融合与跨模态检索。MG$^2$-RAG通过结合轻量级文本解析与实体驱动的视觉定位，构建分层多模态知识图，使文本实体与视觉区域融合为保留原子证据的统一多模态节点。基于该表示，我们引入了多粒度图检索机制，通过聚合密集相似度并在图中传播相关性，支持结构化多跳推理。在四个代表性多模态任务（检索、基于知识视觉问答、推理与分类）上的大量实验表明，MG$^2$-RAG持续达到最先进性能，同时相较于先进基于图框架，图构建开销平均降低43.3倍，成本减少23.9倍。
URL
https://arxiv.org/abs/2604.04969
