Abstract
As video content creation shifts toward long-form narratives, composing short clips into coherent storylines becomes increasingly important. However, prevailing retrieval formulations remain context-agnostic at inference time, prioritizing local semantic alignment while neglecting state and identity consistency. To address this structural limitation, we formalize the task of Consistent Video Retrieval (CVR) and introduce a diagnostic benchmark spanning YouCook2, COIN, and CrossTask. We propose CAST (Context-Aware State Transition), a lightweight, plug-and-play adapter compatible with diverse frozen vision-language embedding spaces. By predicting a state-conditioned residual update ($\Delta$) from visual history, CAST introduces an explicit inductive bias for latent state evolution. Extensive experiments show that CAST improves performance on YouCook2 and CrossTask, remains competitive on COIN, and consistently outperforms zero-shot baselines across diverse foundation backbones. Furthermore, CAST provides a useful reranking signal for black-box video generation candidates (e.g., from Veo), promoting more temporally coherent continuations.
Abstract (translated)
随着视频内容创作转向长篇叙事，将短片组合成连贯的故事线变得越来越重要。然而，现有的检索方法在推理时仍然缺乏上下文感知能力，在优先考虑局部语义对齐的同时忽略了状态和身份的一致性。为了解决这一结构性限制，我们正式定义了“一致视频检索”（Consistent Video Retrieval, CVR）任务，并引入了一个诊断基准，涵盖YouCook2、COIN以及CrossTask数据集。我们提出了一种轻量级且即插即用的适配器CAST（Context-Aware State Transition），它可以与各种冻结的视觉-语言嵌入空间兼容。通过从视觉历史中预测条件状态下的残差更新（$\Delta$），CAST 引入了对潜在状态演化的显式归纳偏置。广泛的实验表明，CAST 在 YouCook2 和 CrossTask 上提高了性能，在 COIN 上保持竞争力，并且在各种基础骨干模型上始终优于零样本基线方法。此外，CAST 还为来自 Veo 等黑盒视频生成候选者的排序提供了有用的信号，促进了更具有时间连贯性的续接。
URL
https://arxiv.org/abs/2603.08648
