Abstract
Long-form video understanding presents significant challenges for interactive retrieval systems, as conventional methods struggle to process extensive video content efficiently. Existing approaches often rely on single models, inefficient storage, unstable temporal search, and context-agnostic reranking, limiting their effectiveness. This paper presents a novel framework to enhance interactive video retrieval through four key innovations: (1) an ensemble search strategy that integrates coarse-grained (CLIP) and fine-grained (BEIT3) models to improve retrieval accuracy, (2) a storage optimization technique that reduces redundancy by selecting representative keyframes via TransNetV2 and deduplication, (3) a temporal search mechanism that localizes video segments using dual queries for start and end points, and (4) a temporal reranking approach that leverages neighboring frame context to stabilize rankings. Evaluated on known-item search and question-answering tasks, our framework demonstrates substantial improvements in retrieval precision, efficiency, and user interpretability, offering a robust solution for real-world interactive video retrieval applications.
Abstract (translated)
长视频理解对于交互式检索系统来说存在重大挑战，因为传统方法难以高效地处理大量视频内容。现有的方法通常依赖单一模型、低效的存储方式、不稳定的时序搜索以及缺乏上下文感知的重排序，这些都限制了它们的效果。本文提出了一种新颖的框架，通过四个关键创新来增强交互式视频检索：(1) 一种集成粗粒度（CLIP）和细粒度（BEIT3）模型的集合搜索策略，以提高检索准确性；(2) 一种存储优化技术，通过选择具有代表性的关键帧并通过TransNetV2去重的方式来减少冗余；(3) 一个使用起始点和结束点双查询来定位视频片段的时序搜索机制；以及 (4) 利用相邻帧上下文以稳定排名的时序重排序方法。在已知项检索和问答任务上进行评估，我们的框架展示了显著提升的检索精度、效率和用户可解释性，为实际交互式视频检索应用提供了稳健的解决方案。
URL
https://arxiv.org/abs/2504.08384
