Abstract
Query performance prediction (QPP) is an important and actively studied information retrieval task, having various applications, such as query reformulation, query expansion, and retrieval system selection, among many others. The task has been primarily studied in the context of text and image retrieval, whereas QPP for content-based video retrieval (CBVR) remains largely underexplored. To this end, we propose the first benchmark for video query performance prediction (VQPP), comprising two text-to-video retrieval datasets and two CBVR systems, respectively. VQPP contains a total of 56K text queries and 51K videos, and comes with official training, validation and test splits, fostering direct comparisons and reproducible results. We explore multiple pre-retrieval and post-retrieval performance predictors, creating a representative benchmark for future exploration of QPP in the video domain. Our results show that pre-retrieval predictors obtain competitive performance, enabling applications before performing the retrieval step. We also demonstrate the applicability of VQPP by employing the best performing pre-retrieval predictor as reward model for training a large language model (LLM) on the query reformulation task via direct preference optimization (DPO). We release our benchmark and code at this https URL.
Abstract (translated)
查询性能预测（QPP）是一项重要的信息检索任务，近年来受到了广泛的研究和关注。它在诸如查询重构、查询扩展以及检索系统选择等众多领域有着各种各样的应用。然而，迄今为止，大部分关于QPP的研究主要集中在文本和图像的检索上，而针对基于内容的视频检索（CBVR）中的查询性能预测问题，则几乎未被深入探索。 为此，我们提出了首个用于视频查询性能预测（VQPP）的基准测试集，该测试包括两个从文本到视频的检索数据集以及两种不同的CBVR系统。我们的VQPP包含总计56,000个文本查询和51,000段视频，并且提供官方的训练、验证及测试分组，以促进直接对比并保证研究结果可重复性。我们探索了多种预检索和后检索性能预测器，为未来在视频领域内QPP的研究提供了具有代表性的基准参考。 我们的结果显示，预检索预测器可以取得相当好的表现，这使得它们能够应用于实际的检索步骤之前的应用场景中。此外，通过使用最佳性能的预检索预测器作为直接偏好优化（DPO）奖励模型来训练大语言模型（LLM），在查询重构任务上进行学习，我们展示了VQPP的实际应用潜力。 我们的基准测试和代码已经发布在这个链接：[此URL]（请将方括号中的文本替换为实际发布的网址）。
URL
https://arxiv.org/abs/2602.17814
