Abstract
Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos partially relevant to a given query. The core challenge lies in learning robust query-video alignment against spurious semantic correlations arising from inherent data uncertainty: 1) query ambiguity, where the query incompletely characterizes the target video and often contains uninformative tokens, and 2) partial video relevance, where abundant query-irrelevant segments introduce contextual noise in cross-modal alignment. Existing methods often focus on enhancing multi-scale clip representations and retrieving the most relevant clip. However, the inherent data uncertainty in PRVR renders them vulnerable to distractor videos with spurious similarities, leading to suboptimal performance. To fill this research gap, we propose Robust Alignment Learning (RAL) framework, which explicitly models the uncertainty in data. Key innovations include: 1) we pioneer probabilistic modeling for PRVR by encoding videos and queries as multivariate Gaussian distributions. This not only quantifies data uncertainty but also enables proxy-level matching to capture the variability in cross-modal correspondences; 2) we consider the heterogeneous informativeness of query words and introduce learnable confidence gates to dynamically weight similarity. As a plug-and-play solution, RAL can be seamlessly integrated into the existing architectures. Extensive experiments across diverse retrieval backbones demonstrate its effectiveness.
Abstract (translated)
部分相关视频检索（PRVR）旨在根据给定的查询检索与之部分相关的未裁剪视频。其核心挑战在于学习稳健的查询-视频对齐，以应对由于数据不确定性而产生的虚假语义关联：1) 查询模糊性，在这种情况下，查询不完全描述目标视频，并且通常包含无信息词汇；2) 部分相关视频，即大量与查询无关的片段引入了跨模态对齐中的上下文噪声。现有的方法往往集中在增强多尺度剪辑表示和检索最相关的剪辑上。然而，在PRVR中固有的数据不确定性使得这些方法容易受到具有虚假相似性的干扰视频的影响，从而导致次优性能。 为填补这一研究空白，我们提出了稳健对齐学习（RAL）框架，该框架明确建模了数据中的不确定性。关键技术创新包括：1) 我们率先为PRVR使用概率模型，并将视频和查询编码为多元高斯分布。这不仅量化了数据的不确定性，还能够通过代理级别匹配捕捉跨模态对应关系的变化；2) 考虑到查询词汇的信息异质性，我们引入可学习的信心门以动态加权相似度。作为即插即用解决方案，RAL可以无缝地集成到现有架构中。在各种检索骨干网络上的广泛实验表明了其有效性。
URL
https://arxiv.org/abs/2509.01383
