Abstract
Composed Video Retrieval (CVR) is a challenging video retrieval task that utilizes multi-modal queries, consisting of a reference video and modification text, to retrieve the desired target video. The core of this task lies in understanding the multi-modal composed query and achieving accurate composed feature learning. Within multi-modal queries, the video modality typically carries richer semantic content compared to the textual modality. However, previous works have largely overlooked the disparity in information density between these two modalities. This limitation can lead to two critical issues: 1) modification subject referring ambiguity and 2) limited detailed semantic focus, both of which degrade the performance of CVR models. To address the aforementioned issues, we propose a novel CVR framework, namely the Hierarchical Uncertainty-aware Disambiguation network (HUD). HUD is the first framework that leverages the disparity in information density between video and text to enhance multi-modal query understanding. It comprises three key components: (a) Holistic Pronoun Disambiguation, (b) Atomistic Uncertainty Modeling, and (c) Holistic-to-Atomistic Alignment. By exploiting overlapping semantics through holistic cross-modal interaction and fine-grained semantic alignment via atomistic-level cross-modal interaction, HUD enables effective object disambiguation and enhances the focus on detailed semantics, thereby achieving precise composed feature learning. Moreover, our proposed HUD is also applicable to the Composed Image Retrieval (CIR) task and achieves state-of-the-art performance across three benchmark datasets for both CVR and CIR tasks. The codes are available on this https URL.
Abstract (translated)
组成视频检索（Composed Video Retrieval，CVR）是一项具有挑战性的视频检索任务，它利用包含参考视频和修改文本的多模态查询来检索所需的靶标视频。这项任务的核心在于理解多模态组合查询并实现准确的组合特征学习。在多模态查询中，视频模式通常携带比文本模式更丰富的语义内容。然而，先前的工作大多忽视了这两种模式之间信息密度差异的问题。这一限制可能导致两个关键问题：1）修改对象指代模糊性；2）有限的具体语义关注点，这些问题都会降低CVR模型的性能。 为解决上述问题，我们提出了一种新颖的CVR框架——分层不确定性感知消歧网络（Hierarchical Uncertainty-aware Disambiguation network，HUD）。HUD是第一个利用视频和文本信息密度差异来增强多模态查询理解的框架。它包含三个关键组成部分：(a) 全局代词消歧；(b) 原子级不确定性建模；(c) 全局到原子级对齐。通过全局跨模式交互利用重叠语义，并通过原子级跨模式交互实现精细语义对齐，HUD能够有效进行对象消歧并增强对具体语义的关注点，从而实现精确的组合特征学习。 此外，我们提出的HUD框架也适用于组成图像检索（Composed Image Retrieval，CIR）任务，在三个基准数据集上均取得了在CVR和CIR任务上的最佳性能。相关代码可在此链接访问：[https URL]。
URL
https://arxiv.org/abs/2512.02792
