Abstract
Multimodal large language models (MLLMs) have shown strong performance on offline video understanding, but most are limited to offline inference or have weak online reasoning, making multi-turn interaction over continuously arriving video streams difficult. Existing streaming methods typically use an interleaved perception-generation paradigm, which prevents concurrent perception and generation and leads to early memory decay as streams grow, hurting long-range dependency modeling. We propose Think While Watching, a memory-anchored streaming video reasoning framework that preserves continuous segment-level memory during multi-turn interaction. We build a three-stage, multi-round chain-of-thought dataset and adopt a stage-matched training strategy, while enforcing strict causality through a segment-level streaming causal mask and streaming positional encoding. During inference, we introduce an efficient pipeline that overlaps watching and thinking and adaptively selects the best attention backend. Under both single-round and multi-round streaming input protocols, our method achieves strong results. Built on Qwen3-VL, it improves single-round accuracy by 2.6% on StreamingBench and by 3.79% on OVO-Bench. In the multi-round setting, it maintains performance while reducing output tokens by 56%. Code is available at: this https URL
Abstract (translated)
多模态大型语言模型（MLLMs）在离线视频理解方面表现出色，但大多数都局限于离线推理或在线推理能力较弱，这使得处理连续到来的视频流中的多次交互变得困难。现有的流式方法通常采用交错感知-生成范式，这种模式阻止了并发感知与生成，并随着数据流的增长导致早期记忆衰退，影响长程依赖建模。 为此，我们提出了“边看边思考”（Think While Watching），这是一个以记忆为中心的流式视频推理框架，在多轮交互中保留连续的片段级记忆。我们建立了一个三阶段、多回合的思想链条数据集，并采用匹配阶段的训练策略，同时通过片段级别的流因果掩码和位置编码来强制执行严格的因果关系。 在推断过程中，我们引入了一种高效的流水线方法，该方法重叠了“看”与“思考”的过程，并根据需要自适应地选择最佳注意力后端。无论是单回合还是多回合的流式输入协议下，我们的方法都能取得优异的成绩。基于Qwen3-VL构建，它在StreamingBench上的单轮精度提高了2.6%，在OVO-Bench上提高了3.79%。在多轮设置中，它可以保持性能的同时减少了输出令牌数56%。 代码可在以下网址获取：[此链接](this https URL)
URL
https://arxiv.org/abs/2603.11896
