Abstract
Existing Multimodal Large Language Models (MLLMs) often suffer from hallucinations in long video understanding (LVU), primarily due to the imbalance between textual and visual tokens. Observing that MLLMs handle short visual inputs well, recent LVU works alleviate hallucinations by automatically parsing the vast visual data into manageable segments that can be effectively processed by MLLMs. SFT-based tool-calling methods can serve this purpose, but they typically require vast amounts of fine-grained, high-quality data and suffer from constrained tool-calling trajectories. We propose a novel VideoTIR that leverages Reinforcement Learning (RL) to encourage proper usage of comprehensive multi-level toolkits for efficient long video understanding. VideoTIR explores both Zero-RL and SFT cold-starting to enable MLLMs to retrieve and focus on meaningful video segments/images/regions, enhancing long video understanding both accurately and efficiently. To reduce redundant tool-calling, we propose Toolkit Action Grouped Policy Optimization (TAGPO), which enhances the efficiency of the calling process through stepwise reward assignment and reuse of failed rollouts. Additionally, we develop a sandbox-based trajectory synthesis framework to generate high-quality trajectories data. Extensive experiments on three long-video QA benchmarks demonstrate the effectiveness and efficiency of our method.
Abstract (translated)
现有多模态大语言模型（MLLMs）在长视频理解（LVU）中常因文本与视觉标记的失衡而产生幻觉。鉴于MLLMs能有效处理短视觉输入，近期LVU研究通过自动解析海量视觉数据为可管理片段来缓解幻觉，从而让MLLMs有效处理。基于监督微调（SFT）的工具调用方法虽可达成此目的，但通常需要大量细粒度高质量数据，且受限于工具调用轨迹。我们提出一种新颖的VideoTIR方法，利用强化学习（RL）促进对多层次综合工具集的合理使用，以实现高效长视频理解。VideoTIR探索零样本RL与SFT冷启动两种方式，使MLLMs能检索并聚焦于有意义的视频片段/图像/区域，从而更准确高效地增强长视频理解。为减少冗余工具调用，我们提出工具操作组策略优化（TAGPO），通过逐步奖励分配与失败轨迹复用提升调用效率。此外，我们开发了基于沙箱的轨迹合成框架以生成高质量轨迹数据。在三个长视频问答基准测试上的大量实验验证了我们方法的有效性与高效性。
URL
https://arxiv.org/abs/2603.25021
