Abstract
Online Video Large Language Models (VideoLLMs) play a critical role in supporting responsive, real-time interaction. Existing methods focus on streaming perception, lacking a synchronized logical reasoning stream. However, directly applying test-time scaling methods incurs unacceptable response latency. To address this trade-off, we propose Video Streaming Thinking (VST), a novel paradigm for streaming video understanding. It supports a thinking while watching mechanism, which activates reasoning over incoming video clips during streaming. This design improves timely comprehension and coherent cognition while preserving real-time responsiveness by amortizing LLM reasoning latency over video playback. Furthermore, we introduce a comprehensive post-training pipeline that integrates VST-SFT, which structurally adapts the offline VideoLLM to causal streaming reasoning, and VST-RL, which provides end-to-end improvement through self-exploration in a multi-turn video interaction environment. Additionally, we devise an automated training-data synthesis pipeline that uses video knowledge graphs to generate high-quality streaming QA pairs, with an entity-relation grounded streaming Chain-of-Thought to enforce multi-evidence reasoning and sustained attention to the video stream. Extensive evaluations show that VST-7B performs strongly on online benchmarks, e.g. 79.5% on StreamingBench and 59.3% on OVO-Bench. Meanwhile, VST remains competitive on offline long-form or reasoning benchmarks. Compared with Video-R1, VST responds 15.7 times faster and achieves +5.4% improvement on VideoHolmes, demonstrating higher efficiency and strong generalization across diverse video understanding tasks. Code, data, and models will be released at this https URL.
Abstract (translated)
在线视频大型语言模型（VideoLLMs）在支持响应式、实时交互中扮演着关键角色。现有方法主要集中在流媒体感知上，缺乏同步的逻辑推理过程。然而，直接应用测试时间缩放方法会导致无法接受的响应延迟。为解决这一权衡问题，我们提出了视频流思考（VST），这是一种新的用于视频理解的流媒体范式。它支持边看边想机制，在流媒体期间激活对传入视频片段的推理。这种设计通过将大型语言模型的推理延迟分摊到视频播放中来提高及时理解和连贯的认知，并保持实时响应性。 此外，我们还引入了一个全面的后训练流水线，该流水线整合了VST-SFT和VST-RL。VST-SFT结构性地将离线VideoLLM适应为因果流推理，而VST-RL则通过在多回合视频互动环境中自我探索提供端到端改进。我们还设计了一个自动化的训练数据合成流水线，该流水线使用视频知识图来生成高质量的流媒体问答对，并采用基于实体关系的连贯思考链来强制执行多重证据推理和对视频流的持续关注。 广泛的评估表明，VST-7B在在线基准测试（如StreamingBench上的79.5%和OVO-Bench上的59.3%）中表现强劲。同时，在离线长格式或推理基准上，VST也保持了竞争力。与Video-R1相比，VST的响应速度快了15.7倍，并在VideoHolmes上实现了+5.4%的改进，展示了更高的效率和跨多样视频理解任务的强大泛化能力。 代码、数据和模型将在以下网址发布：[https URL]。
URL
https://arxiv.org/abs/2603.12262
