Abstract
Partially Relevant Video Retrieval (PRVR) aims to retrieve the target video that is partially relevant to the text query. The primary challenge in PRVR arises from the semantic asymmetry between textual and visual modalities, as videos often contain substantial content irrelevant to the query. Existing methods coarsely align paired videos and text queries to construct the semantic space, neglecting the critical cross-modal dual nature inherent in this task: inter-sample correlation and intra-sample redundancy. To this end, we propose a novel PRVR framework to systematically exploit these two characteristics. Our framework consists of three core modules. First, the Inter Correlation Enhancement (ICE) module captures inter-sample correlation by identifying semantically similar yet unpaired text queries and video moments, combining them to form pseudo-positive pairs for more robust semantic space construction. Second, the Intra Redundancy Mining (IRM) module mitigates intra-sample redundancy by mining redundant video moment features and treating them as hard negative samples, thereby encouraging the model to learn more discriminative representations. Finally, to reinforce these modules, we introduce the Temporal Coherence Prediction (TCP) module, which enhances feature discrimination by training the model to predict the original temporal order of randomly shuffled video frames and moments. Extensive experiments on three datasets demonstrate the superiority of our approach compared to previous methods, achieving state-of-the-art results.
Abstract (translated)
部分相关视频检索（PRVR）的目标是从文本查询中提取出与之部分相关的目标视频。在PRVR中，主要挑战来自于文字和视觉模态之间的语义不对称性：由于视频通常包含大量无关查询内容的信息，因此难以精确匹配。 现有的方法通过粗略地对齐配对的视频和文本查询来构建语义空间，这忽略了任务内在的关键跨模态二元特性：样本间的关联性和样本内的冗余性。为此，我们提出了一种新的PRVR框架，旨在系统地利用这两种特征。我们的框架包括三个核心模块： 1. **交互相关增强（ICE）** 模块通过识别语义相似但未配对的文本查询和视频片段来捕捉样本间的关联性，并将它们组合成伪正向对以构建更稳健的语义空间。 2. **内部冗余挖掘（IRM）** 模块通过挖掘视频片段特征中的冗余信息并将其视为难负例样本来缓解样本内的冗余问题，从而鼓励模型学习更具区分性的表示方法。 3. 为了强化这些模块的效果，我们引入了**时间一致性预测（TCP）** 模块。该模块通过对随机打乱的视频帧和片段进行时间顺序的预测来增强特征的区分性，以此训练模型。 在三个数据集上的广泛实验结果表明，与先前的方法相比，我们的方法具有明显的优势，并取得了最新的研究成果。
URL
https://arxiv.org/abs/2504.19637
