Abstract
Multimodal document retrieval systems enable information access across text, images, and layouts, benefiting various domains like document-based question answering, report analysis, and interactive content summarization. Rerankers improve retrieval precision by reordering retrieved candidates. However, current multimodal reranking methods remain underexplored, with significant room for improvement in both training strategies and overall effectiveness. Moreover, the lack of explicit reasoning makes it difficult to analyze and optimize these methods further. In this paper, We propose MM-R5, a MultiModal Reasoning-Enhanced ReRanker via Reinforcement Learning for Document Retrieval, aiming to provide a more effective and reliable solution for multimodal reranking tasks. MM-R5 is trained in two stages: supervised fine-tuning (SFT) and reinforcement learning (RL). In the SFT stage, we focus on improving instruction-following and guiding the model to generate complete and high-quality reasoning chains. To support this, we introduce a novel data construction strategy that produces rich, high-quality reasoning data. In the RL stage, we design a task-specific reward framework, including a reranking reward tailored for multimodal candidates and a composite template-based reward to further refine reasoning quality. We conduct extensive experiments on MMDocIR, a challenging public benchmark spanning multiple domains. MM-R5 achieves state-of-the-art performance on most metrics and delivers comparable results to much larger models on the remaining ones. Moreover, compared to the best retrieval-only method, MM-R5 improves recall@1 by over 4%. These results validate the effectiveness of our reasoning-enhanced training pipeline.
Abstract (translated)
多模态文档检索系统能够跨越文本、图像和版面布局，为文档问答、报告分析和互动内容总结等众多领域提供信息访问服务。重排序器通过重新排列检索到的候选对象来提高检索精度。然而，目前的多模态重排序方法仍然研究不足，在训练策略及整体有效性方面还有很大的改进空间。此外，缺乏明确的推理机制使得这些方法难以进一步分析和优化。 本文提出了一种名为MM-R5的方法，这是一种通过强化学习增强多模态推理能力的文档检索重排序器（MultiModal Reasoning-Enhanced ReRanker via Reinforcement Learning for Document Retrieval），旨在为多模态重排序任务提供更有效且可靠的解决方案。MM-R5分为两个训练阶段：监督微调（SFT）和强化学习（RL）。在SFT阶段，我们专注于改进指令遵循能力，并引导模型生成完整而高质量的推理链。为此，我们引入了一种新颖的数据构建策略，以产生丰富、高质量的推理数据。 在RL阶段，我们设计了一个特定任务的奖励框架，其中包括为多模态候选对象定制的重排序奖励以及进一步精炼推理质量的基于复合模板的奖励。我们在MMDocIR上进行了广泛的实验，这是一个跨越多个领域的具有挑战性的公共基准测试集。MM-R5在大多数指标上达到了最先进的性能，并且对于剩余的一些指标来说，其表现与更大规模的模型相当。 此外，相较于仅限于检索的方法，MM-R5提升了超过4%的召回率@1。这些结果验证了我们增强推理能力的训练管道的有效性。
URL
https://arxiv.org/abs/2506.12364
