Abstract
Text-Video Retrieval aims to find the most relevant text (or video) candidate given a video (or text) query from large-scale online databases. Recent work leverages multi-modal large language models (MLLMs) to improve retrieval, especially for long or complex query-candidate pairs. However, we observe that the naive application of MLLMs, i.e., retrieval based on candidate likelihood, introduces candidate prior bias, favoring candidates with inherently higher priors over those more relevant to the query. To this end, we propose a novel retrieval framework, Bidirectional Likelihood Estimation with MLLM (BLiM), which leverages both query and candidate likelihoods by training the model to generate text from a given video as well as video features from a given text. Furthermore, we introduce Candidate Prior Normalization (CPN), a simple yet effective training-free score calibration module designed to mitigate candidate prior bias in candidate likelihood. On four Text-Video Retrieval benchmarks, our BLiM equipped with CPN outperforms previous state-of-the-art models by 6.4 R@1 on average, effectively alleviating candidate prior bias and emphasizing query-candidate relevance. Our in-depth analysis across various multi-modal tasks beyond retrieval highlights the broad applicability of CPN which enhances visual understanding by reducing reliance on textual priors. Code is available at this https URL.
Abstract (translated)
文本-视频检索的目标是从大规模在线数据库中，根据给定的视频（或文本）查询找到最相关的文本（或视频）。最近的研究利用了多模态大型语言模型（MLLMs）来改进检索性能，特别是对于长或者复杂的查询候选对。然而，我们观察到直接应用MLLMs，即基于候选可能性进行检索的方法，引入了候选先验偏见，倾向于选择那些本身具有更高先验值的候选对象而非与查询更相关的对象。 为此，我们提出了一个新颖的检索框架——双向似然估计多模态模型（BLiM），该框架利用了查询和候选的概率，通过训练模型从给定视频生成文本以及从给定文本生成视频特征。此外，我们引入了一种简单而有效的无需训练的评分校准模块——候选先验归一化（CPN），专门用于缓解候选概率中的候选先验偏见问题。 在四个文本-视频检索基准测试中，配备有CPN的BLiM模型平均提高了6.4个R@1点的性能，有效减轻了候选先验偏见，并强调了查询-候选的相关性。我们对各种多模态任务（超出检索范围）进行了深入分析，表明CPN具有广泛的应用潜力，通过减少对文本先验的依赖来增强视觉理解能力。 代码可在[此处](https://this.is.a.mock.url)获取。（注：实际链接应替换为真实提供的URL地址。）
URL
https://arxiv.org/abs/2507.23284
