Abstract
Given a text query, partially relevant video retrieval (PRVR) aims to retrieve untrimmed videos containing relevant moments, wherein event modeling is crucial for partitioning the video into smaller temporal events that partially correspond to the text. Previous methods typically segment videos into a fixed number of equal-length clips, resulting in ambiguous event boundaries. Additionally, they rely on mean pooling to compute event representations, inevitably introducing undesired misalignment. To address these, we propose an Uneven Event Modeling (UEM) framework for PRVR. We first introduce the Progressive-Grouped Video Segmentation (PGVS) module, to iteratively formulate events in light of both temporal dependencies and semantic similarity between consecutive frames, enabling clear event boundaries. Furthermore, we also propose the Context-Aware Event Refinement (CAER) module to refine the event representation conditioned the text's cross-attention. This enables event representations to focus on the most relevant frames for a given text, facilitating more precise text-video alignment. Extensive experiments demonstrate that our method achieves state-of-the-art performance on two PRVR benchmarks.
Abstract (translated)
给定一个文本查询，部分相关视频检索（PRVR）的目标是从未修剪的视频中检索包含相关内容片段的视频。在此过程中，事件建模对于将视频分割成较小的时间段事件至关重要，这些时间段与文本部分内容相对应。以往的方法通常会将视频分割为固定数量、等长的片段，这会导致模糊的事件边界。此外，它们依赖于均值池化来计算事件表示，不可避免地引入了不必要的错位。 为了应对这些问题，我们提出了一个不均匀事件建模（UEM）框架用于PRVR。首先，我们介绍了渐进分组视频分割（PGVS）模块，该模块可以迭代地根据连续帧之间的时序依赖性和语义相似性来制定事件，从而实现清晰的事件边界。此外，我们还提出了一种基于上下文感知事件细化（CAER）模块，以条件于文本跨注意力的方式对事件表示进行精炼。这使得事件表示能够专注于与给定文本最相关的帧上，有助于更精确地进行文本和视频之间的对齐。 广泛的实验表明，我们的方法在两个PRVR基准测试中达到了最先进的性能水平。
URL
https://arxiv.org/abs/2506.00891
