Abstract
Video understanding requires not only recognizing visual content but also performing temporally grounded, multi-step reasoning over long and noisy observations. We propose Process-of-Thought (PoT) Reasoning for Videos, a framework that makes the reasoning process explicit by structuring video inference into a sequence of lightweight, verifiable steps. PoT interleaves (i) temporal evidence selection, (ii) step-wise state updates, and (iii) constrained answer synthesis, enabling the model to progressively refine hypotheses while maintaining traceability to video evidence. The framework is designed to be model-agnostic and can be plugged into existing vision-language backbones, supporting both closed-book reasoning and evidence-augmented reasoning with external tools. We further introduce a unified representation for PoT traces that aligns intermediate decisions with temporal segments, which improves robustness to distractors and reduces hallucinated explanations. Extensive experiments on standard video reasoning tasks demonstrate that PoT consistently improves factual correctness and temporal grounding, while providing interpretable reasoning traces for diagnosis and downstream use.
Abstract (translated)
视频理解不仅需要识别视觉内容，还需要对长时间且可能存在噪声的观察进行基于时间顺序的多步骤推理。为此，我们提出了用于视频的“过程思维”（PoT）推理框架，该框架通过将视频推断结构化为一系列轻量级、可验证的步骤来使推理过程透明化。PoT 交替执行以下操作：(i) 时间证据选择；(ii) 分步状态更新；以及 (iii) 约束性答案合成，这使得模型能够逐步细化假设并保持对视频证据的追溯能力。 该框架是模型无关的，并且可以插入现有的视觉-语言骨干网络中，支持闭卷推理和借助外部工具增强证据的推理。我们进一步引入了一种统一表示法来为 PoT 跟踪提供时间片段与中间决策之间的对齐，从而增强了对干扰物的鲁棒性并减少了虚构解释的数量。 在标准视频推理任务上的广泛实验表明，PoT 在提高事实准确性、基于时间顺序的基础以及提供可解释的推理跟踪方面始终如一地带来了改进，这些跟踪可用于诊断和下游应用。
URL
https://arxiv.org/abs/2602.07689
