Abstract
Composed Video Retrieval (CoVR) aims to find a target video given a reference video and a textual modification. Prior work assumes the modification text fully specifies the visual changes, overlooking after-effects and implicit consequences (e.g., motion, state transitions, viewpoint or duration cues) that emerge from the edit. We argue that successful CoVR requires reasoning about these after-effects. We introduce a reasoning-first, zero-shot approach that leverages large multimodal models to (i) infer causal and temporal consequences implied by the edit, and (ii) align the resulting reasoned queries to candidate videos without task-specific finetuning. To evaluate reasoning in CoVR, we also propose CoVR-Reason, a benchmark that pairs each (reference, edit, target) triplet with structured internal reasoning traces and challenging distractors that require predicting after-effects rather than keyword matching. Experiments show that our zero-shot method outperforms strong retrieval baselines on recall at K and particularly excels on implicit-effect subsets. Our automatic and human analysis confirm higher step consistency and effect factuality in our retrieved results. Our findings show that incorporating reasoning into general-purpose multimodal models enables effective CoVR by explicitly accounting for causal and temporal after-effects. This reduces dependence on task-specific supervision, improves generalization to challenging implicit-effect cases, and enhances interpretability of retrieval outcomes. These results point toward a scalable and principled framework for explainable video search. The model, code, and benchmark are available at this https URL.
Abstract (translated)
组合视频检索（CoVR）旨在根据参考视频和文本修改描述来查找目标视频。以往工作假设修改文本能完全指定视觉变化，却忽略了编辑操作所产生的后续效应与隐含后果（例如运动、状态转换、视角或时长线索等）。我们认为，成功的CoVR必须对这些后续效应进行推理。我们提出一种推理优先的零样本方法，利用大语言多模态模型：（i）推断编辑所隐含的因果与时序后果；（ii）将推理生成的查询与候选视频对齐，而无需任务特定的微调。为评估CoVR中的推理能力，我们还构建了CoVR-Reason基准，该基准为每个（参考视频、编辑描述、目标视频）三元组配对了结构化内部推理轨迹，以及需要预测后续效应而非关键词匹配的挑战性干扰项。实验表明，我们的零样本方法在K值召回率上优于强检索基线，尤其在隐含效应子集上表现突出。自动与人工分析均证实，我们检索结果具有更高的步骤一致性和效应事实性。研究结果表明，将推理融入通用多模态模型，可通过显式考虑因果与时序后续效应来实现有效的CoVR。这降低了对任务特定监督的依赖，提升了对挑战性隐含效应案例的泛化能力，并增强了检索结果的可解释性。这些成果指向了一个可扩展且原则明确的视频可解释搜索框架。相关模型、代码与基准已通过此链接公开。
URL
https://arxiv.org/abs/2603.20190
