Abstract
Traditional video retrieval benchmarks focus on matching precise descriptions to closed video pools, failing to reflect real-world searches characterized by fuzzy, multi-dimensional memories on the open web. We present \textbf{RVMS-Bench}, a comprehensive system for evaluating real-world video memory search. It consists of \textbf{1,440 samples} spanning \textbf{20 diverse categories} and \textbf{four duration groups}, sourced from \textbf{real-world open-web videos}. RVMS-Bench utilizes a hierarchical description framework encompassing \textbf{Global Impression, Key Moment, Temporal Context, and Auditory Memory} to mimic realistic multi-dimensional search cues, with all samples strictly verified via a human-in-the-loop protocol. We further propose \textbf{RACLO}, an agentic framework that employs abductive reasoning to simulate the human ``Recall-Search-Verify'' cognitive process, effectively addressing the challenge of searching for videos via fuzzy memories in the real world. Experiments reveal that existing MLLMs still demonstrate insufficient capabilities in real-world Video Retrieval and Moment Localization based on fuzzy memories. We believe this work will facilitate the advancement of video retrieval robustness in real-world unstructured scenarios.
Abstract (translated)
传统的视频检索基准主要集中在将精确描述与封闭的视频库进行匹配，未能反映开放网络中由模糊、多维度记忆特征的真实世界搜索。我们提出了**RVMS-Bench**，这是一个用于评估现实世界视频记忆搜索的全面系统。该系统包含**1,440个样本**，涵盖**20种不同的类别**和**四个持续时间组别**，所有数据均来自**真实世界的开放网络视频**。RVMS-Bench利用一个分层描述框架，包括**全局印象、关键时刻、时间背景以及听觉记忆**，以模仿现实世界中多维度的搜索线索，并通过人机交互协议对所有样本进行严格验证。 我们进一步提出了**RACLO**，这是一种代理框架，它使用溯因推理来模拟人类的“回忆-搜索-验证”认知过程，有效地解决了基于模糊记忆在现实环境中寻找视频这一挑战。实验结果显示，现有的多模态语言模型（MLLMs）在基于模糊记忆的真实世界视频检索和时刻定位方面仍表现出不足的能力。 我们认为这项工作将有助于推动在实际无结构场景中视频检索鲁棒性的进步。
URL
https://arxiv.org/abs/2602.10159
