Abstract
Reinforcement Learning (RL) is crucial for empowering VideoLLMs with complex spatiotemporal reasoning. However, current RL paradigms predominantly rely on random data shuffling or naive curriculum strategies based on scalar difficulty metrics. We argue that scalar metrics fail to disentangle two orthogonal challenges in video understanding: Visual Temporal Perception Load and Cognitive Reasoning Depth. To address this, we propose VideoCuRL, a novel framework that decomposes difficulty into these two axes. We employ efficient, training-free proxies, optical flow and keyframe entropy for visual complexity, Calibrated Surprisal for cognitive complexity, to map data onto a 2D curriculum grid. A competence aware Diagonal Wavefront strategy then schedules training from base alignment to complex reasoning. Furthermore, we introduce Dynamic Sparse KL and Structured Revisiting to stabilize training against reward collapse and catastrophic forgetting. Extensive experiments show that VideoCuRL surpasses strong RL baselines on reasoning (+2.5 on VSI-Bench) and perception (+2.9 on VideoMME) tasks. Notably, VideoCuRL eliminates the prohibitive inference overhead of generation-based curricula, offering a scalable solution for robust video post-training.
Abstract (translated)
强化学习（RL）对于赋予VideoLLMs复杂时空推理能力至关重要。然而，目前的RL范式主要依赖于随机数据洗牌或基于标量难度指标的简单课程策略。我们认为，标量度量无法分离视频理解中的两个正交挑战：视觉时间感知负载和认知推理深度。为解决这一问题，我们提出了VideoCuRL，这是一个将困难性分解为这两轴的新框架。对于视觉复杂性的训练免费代理，我们使用了高效的光学流和关键帧熵；而对于认知复杂性，我们采用了校准的意外度量，并将其映射到一个2D课程网格上。之后采用能力感知对角波前策略从基本对齐训练至复杂的推理。此外，为了在面对奖励崩溃和灾难性遗忘时稳定训练，我们引入了动态稀疏KL和结构化回访机制。广泛的实验表明，在推理（VSI-Bench上的+2.5）和感知（VideoMME上的+2.9）任务上，VideoCuRL超越了强大的RL基线模型。值得注意的是，VideoCuRL消除了生成式课程的繁重推断开销，提供了一个可扩展的解决方案以增强视频训练后的稳健性。
URL
https://arxiv.org/abs/2601.00887
