Abstract
Partially relevant video retrieval (PRVR) is a practical yet challenging task in text-to-video retrieval, where videos are untrimmed and contain much background content. The pursuit here is of both effective and efficient solutions to capture the partial correspondence between text queries and untrimmed videos. Existing PRVR methods, which typically focus on modeling multi-scale clip representations, however, suffer from content independence and information redundancy, impairing retrieval performance. To overcome these limitations, we propose a simple yet effective approach with active moment discovering (AMDNet). We are committed to discovering video moments that are semantically consistent with their queries. By using learnable span anchors to capture distinct moments and applying masked multi-moment attention to emphasize salient moments while suppressing redundant backgrounds, we achieve more compact and informative video representations. To further enhance moment modeling, we introduce a moment diversity loss to encourage different moments of distinct regions and a moment relevance loss to promote semantically query-relevant moments, which cooperate with a partially relevant retrieval loss for end-to-end optimization. Extensive experiments on two large-scale video datasets (\ie, TVR and ActivityNet Captions) demonstrate the superiority and efficiency of our AMDNet. In particular, AMDNet is about 15.5 times smaller (\#parameters) while 6.0 points higher (SumR) than the up-to-date method GMMFormer on TVR.
Abstract (translated)
部分相关视频检索（PRVR）是文本到视频检索中的一项实用且具有挑战性的任务，其中的视频未经过修剪，并包含大量背景内容。该领域的目标在于寻找既有效又高效的解决方案，以捕捉文字查询与未修剪视频之间的局部对应关系。然而，现有的PRVR方法通常侧重于多尺度片段表示建模，但这些方法往往受到内容独立性和信息冗余的影响，从而损害了检索性能。 为了克服这些限制，我们提出了一种简单而有效的主动时刻发现（AMDNet）方法。我们的目标是发现与查询语义一致的视频时刻。通过使用可学习的时间间隔锚点来捕捉不同的时刻，并应用掩码多时刻注意力机制以强调显著性时刻同时抑制冗余背景，我们可以实现更为紧凑和信息丰富的视频表示。 为进一步增强时刻建模，我们引入了两种损失函数：一种是多样性损失（moment diversity loss），鼓励不同区域中不相同的时刻；另一种是相关性损失（moment relevance loss），促进与查询语义相关的时刻。这两种损失协同工作，并与部分相关检索损失共同作用于端到端优化。 在两个大规模视频数据集（即TVR和ActivityNet Captions）上的广泛实验表明，我们的AMDNet方法具有优越性和效率。具体而言，在TVR数据集中，AMDNet比当前最先进的GMMFormer方法小约15.5倍（参数数量），同时性能高出6.0分（SumR）。
URL
https://arxiv.org/abs/2504.10920
