Video_Retrieval
-
Omni-Embed-Nemotron: A Unified Multimodal Retrieval Model for Text, Image, Audio, and Video
2025-10-03 19:29:50Mengyao Xu, Wenfei Zhou, Yauhen Babakhin, Gabriel Moreira, Ronay Ak, Radek Osmulski, Bo Liu, Even Oldridge, Benedikt SchiffererAbstract
We present Omni-Embed-Nemotron, a unified multimodal retrieval embedding model developed to handle the increasing complexity of real-world information needs. While Retrieval-Augmented Generation (RAG) has significantly advanced language models by incorporating external knowledge, existing text-based retrievers rely on clean, structured input and struggle with the visually and semantically rich content found in real-world documents such as PDFs, slides, or videos. Recent work such as ColPali has shown that preserving document layout using image-based representations can improve retrieval quality. Building on this, and inspired by the capabilities of recent multimodal models such as Qwen2.5-Omni, we extend retrieval beyond text and images to also support audio and video modalities. Omni-Embed-Nemotron enables both cross-modal (e.g., text - video) and joint-modal (e.g., text - video+audio) retrieval using a single model. We describe the architecture, training setup, and evaluation results of Omni-Embed-Nemotron, and demonstrate its effectiveness in text, image, and video retrieval.
Abstract (translated)
我们介绍了Omni-Embed-Nemotron，这是一种统一的多模态检索嵌入模型，旨在处理日益复杂的真实世界信息需求。虽然检索增强生成（RAG）通过整合外部知识显著提升了语言模型的能力，但现有的基于文本的检索器依赖于干净、结构化的输入，并且在处理如PDF文件、幻灯片或视频中发现的视觉和语义丰富的内容时显得力不从心。最近的研究，例如ColPali，表明使用图像表示来保留文档布局可以提升检索质量。在此基础上，并受到近期多模态模型（如Qwen2.5-Omni）能力的启发，我们将检索范围扩展到不仅包括文本和图像，还包括音频和视频模式。Omni-Embed-Nemotron能够同时支持跨模态（例如，文本与视频）和联合模态（例如，文本、视频加音频）检索，仅使用单一模型即可实现。 本文描述了Omni-Embed-Nemotron的架构、训练设置及评估结果，并展示了其在文本、图像和视频检索中的有效性。
URL
https://arxiv.org/abs/2510.03458
PDF
-
Panorama: Fast-Track Nearest Neighbors
2025-10-01 06:38:45Vansh Ramani, Alexis Schlomer, Akash Nayar, Panagiotis Karras, Sayan Ranu, Jignesh M. PatelAbstract
Approximate Nearest-Neighbor Search (ANNS) efficiently finds data items whose embeddings are close to that of a given query in a high-dimensional space, aiming to balance accuracy with speed. Used in recommendation systems, image and video retrieval, natural language processing, and retrieval-augmented generation (RAG), ANNS algorithms such as IVFPQ, HNSW graphs, Annoy, and MRPT utilize graph, tree, clustering, and quantization techniques to navigate large vector spaces. Despite this progress, ANNS systems spend up to 99\% of query time to compute distances in their final refinement phase. In this paper, we present PANORAMA, a machine learning-driven approach that tackles the ANNS verification bottleneck through data-adaptive learned orthogonal transforms that facilitate the accretive refinement of distance bounds. Such transforms compact over 90\% of signal energy into the first half of dimensions, enabling early candidate pruning with partial distance computations. We integrate PANORAMA into state-of-the-art ANNS methods, namely IVFPQ/Flat, HNSW, MRPT, and Annoy, without index modification, using level-major memory layouts, SIMD-vectorized partial distance computations, and cache-aware access patterns. Experiments across diverse datasets -- from image-based CIFAR-10 and GIST to modern embedding spaces including OpenAI's Ada 2 and Large 3 -- demonstrate that PANORAMA affords a 2--30$\times$ end-to-end speedup with no recall loss.
Abstract (translated)
近似最近邻搜索（ANNS）能够在高维空间中高效地找到与给定查询嵌入接近的数据项，旨在平衡准确性和速度。该技术广泛应用于推荐系统、图像和视频检索、自然语言处理以及增强检索的生成模型（RAG）。ANNS算法如IVFPQ、HNSW图、Annoy和MRPT利用图、树、聚类及量化等方法在庞大的向量空间中导航。尽管取得了这些进展，ANNS系统仍需花费高达99%的查询时间来计算最终细化阶段的距离。 本文介绍了PANORAMA，这是一种基于机器学习的方法，通过数据自适应地学习正交变换来解决ANNS验证瓶颈问题，这种方法能够逐步精确距离界限。这些变换将超过90%的能量集中于前半部分维度，使得可以在进行部分距离计算时提前剔除候选对象。 我们将PANORAMA无缝集成到最先进的ANNS方法（包括IVFPQ/Flat、HNSW、MRPT和Annoy）中，无需修改索引，同时利用层级优先内存布局、SIMD向量化部分距离计算以及缓存感知访问模式。跨多种数据集的实验——从基于图像的CIFAR-10和GIST到包括OpenAI的Ada 2和Large 3在内的现代嵌入空间——表明PANORAMA能够在不损失召回率的情况下提供高达2至30倍的整体加速效果。
URL
https://arxiv.org/abs/2510.00566
PDF
-
X-CoT: Explainable Text-to-Video Retrieval via LLM-based Chain-of-Thought Reasoning
2025-09-25 20:39:45Prasanna Reddy Pulakurthi, Jiamian Wang, Majid Rabbani, Sohail Dianat, Raghuveer Rao, Zhiqiang TaoAbstract
Prevalent text-to-video retrieval systems mainly adopt embedding models for feature extraction and compute cosine similarities for ranking. However, this design presents two limitations. Low-quality text-video data pairs could compromise the retrieval, yet are hard to identify and examine. Cosine similarity alone provides no explanation for the ranking results, limiting the interpretability. We ask that can we interpret the ranking results, so as to assess the retrieval models and examine the text-video data? This work proposes X-CoT, an explainable retrieval framework upon LLM CoT reasoning in place of the embedding model-based similarity ranking. We first expand the existing benchmarks with additional video annotations to support semantic understanding and reduce data bias. We also devise a retrieval CoT consisting of pairwise comparison steps, yielding detailed reasoning and complete ranking. X-CoT empirically improves the retrieval performance and produces detailed rationales. It also facilitates the model behavior and data quality analysis. Code and data are available at: this https URL.
Abstract (translated)
当前的文本到视频检索系统主要采用嵌入模型进行特征提取，并通过计算余弦相似度来进行排序。然而，这种设计存在两个局限性：低质量的文本-视频数据对可能会损害检索效果，但这些低质量的数据却难以识别和审查；仅靠余弦相似度无法解释排名结果，从而限制了系统的可解释性。我们提出的问题是能否解释排名结果，以便评估检索模型并检查文本-视频数据？ 针对这些问题，这项工作提出了X-CoT（基于大规模语言模型CoT推理的可解释检索框架），用以替代基于嵌入模型的相似度排序方法。首先，我们在现有的基准测试基础上增加了额外的视频注释，支持语义理解和减少数据偏差。此外，我们设计了一种检索CoT（通过成对比较步骤生成详细的推理和完整的排名）。X-CoT在实践中提高了检索性能，并产生详细的原因说明。它还促进了模型行为和数据质量分析。 代码和数据可以在以下链接获取：[此URL](this https URL)
URL
https://arxiv.org/abs/2509.21559
PDF
-
ConViS-Bench: Estimating Video Similarity Through Semantic Concepts
2025-09-23 17:06:11Benedetta Liberatori, Alessandro Conti, Lorenzo Vaquero, Yiming Wang, Elisa Ricci, Paolo RotaAbstract
What does it mean for two videos to be similar? Videos may appear similar when judged by the actions they depict, yet entirely different if evaluated based on the locations where they were filmed. While humans naturally compare videos by taking different aspects into account, this ability has not been thoroughly studied and presents a challenge for models that often depend on broad global similarity scores. Large Multimodal Models (LMMs) with video understanding capabilities open new opportunities for leveraging natural language in comparative video tasks. We introduce Concept-based Video Similarity estimation (ConViS), a novel task that compares pairs of videos by computing interpretable similarity scores across a predefined set of key semantic concepts. ConViS allows for human-like reasoning about video similarity and enables new applications such as concept-conditioned video retrieval. To support this task, we also introduce ConViS-Bench, a new benchmark comprising carefully annotated video pairs spanning multiple domains. Each pair comes with concept-level similarity scores and textual descriptions of both differences and similarities. Additionally, we benchmark several state-of-the-art models on ConViS, providing insights into their alignment with human judgments. Our results reveal significant performance differences on ConViS, indicating that some concepts present greater challenges for estimating video similarity. We believe that ConViS-Bench will serve as a valuable resource for advancing research in language-driven video understanding.
Abstract (translated)
两个视频相似意味着什么？根据它们所描绘的动作来判断，两个视频可能会显得很相似；但如果依据拍摄地点来进行评估，则可能完全不同。人类在比较视频时会自然地考虑不同的方面，但这一能力尚未被充分研究，并且对于依赖广泛全局相似度分数的模型来说是一个挑战。具备视频理解功能的大规模多模态模型（LMM）为在对比视频任务中利用自然语言带来了新的机会。 我们引入了基于概念的视频相似性估计（ConViS），这是一个新颖的任务，通过计算一组预定义的关键语义概念跨视频对的可解释相似度分数来比较视频。ConViS 允许像人类一样推理关于视频的相似性，并且能够支持如基于概念条件的视频检索等新的应用。 为了支持这一任务，我们还引入了 ConViS-Bench，一个新的基准测试库，它包含精心标注的跨多个领域的视频对。每一对视频都带有概念级别的相似度分数和差异及相似性的文本描述。 此外，我们在 ConViS 上评估了几种最先进的模型，提供了关于它们与人类判断一致程度的见解。我们的结果表明，在 ConViS 任务上存在显著的表现差距，这表明某些概念对于估计视频相似性来说更具挑战性。 我们认为，ConViS-Bench 将成为推动基于语言的视频理解研究的重要资源。
URL
https://arxiv.org/abs/2509.19245
PDF
-
VidCLearn: A Continual Learning Approach for Text-to-Video Generation
2025-09-21 07:34:19Luca Zanchetta, Lorenzo Papa, Luca Maiano, Irene AmeriniAbstract
Text-to-video generation is an emerging field in generative AI, enabling the creation of realistic, semantically accurate videos from text prompts. While current models achieve impressive visual quality and alignment with input text, they typically rely on static knowledge, making it difficult to incorporate new data without retraining from scratch. To address this limitation, we propose VidCLearn, a continual learning framework for diffusion-based text-to-video generation. VidCLearn features a student-teacher architecture where the student model is incrementally updated with new text-video pairs, and the teacher model helps preserve previously learned knowledge through generative replay. Additionally, we introduce a novel temporal consistency loss to enhance motion smoothness and a video retrieval module to provide structural guidance at inference. Our architecture is also designed to be more computationally efficient than existing models while retaining satisfactory generation performance. Experimental results show VidCLearn's superiority over baseline methods in terms of visual quality, semantic alignment, and temporal coherence.
Abstract (translated)
文本到视频生成是生成式人工智能中的一个新兴领域，能够从文字提示中创建出视觉上逼真且语义准确的视频。尽管当前模型在视觉质量和与输入文本的一致性方面取得了令人印象深刻的成绩，但它们通常依赖于静态知识，这使得难以在不重新训练的情况下将新数据纳入系统中。为了解决这一限制，我们提出了VidCLearn，这是一种基于扩散方法的连续学习框架用于文本到视频生成。 VidCLearn采用了一种师生架构，在这种架构下，学生模型会随着新的文字-视频对逐渐更新，而教师模型则通过生成重播来帮助保留之前学到的知识。此外，我们还引入了一种新颖的时间一致性损失函数以增强运动的平滑度，并加入了一个视频检索模块用于在推理时提供结构指导。 我们的框架设计为比现有模型计算效率更高，同时仍能保持满意的生成性能。实验结果表明，与基准方法相比，VidCLearn在视觉质量、语义一致性和时间连贯性方面均表现出优越性。
URL
https://arxiv.org/abs/2509.16956
PDF
-
Captioning for Text-Video Retrieval via Dual-Group Direct Preference Optimization
2025-09-20 07:36:53Ji Soo Lee, Byungoh Ko, Jaewon Cho, Howoong Lee, Jaewoon Byun, Hyunwoo J. KimAbstract
In text-video retrieval, auxiliary captions are often used to enhance video understanding, bridging the gap between the modalities. While recent advances in multi-modal large language models (MLLMs) have enabled strong zero-shot caption generation, we observe that such captions tend to be generic and indistinguishable across visually similar videos, limiting their utility for fine-grained retrieval. Moreover, conventional captioning approaches are typically evaluated using language generation metrics, such as BLEU, which are not typically tailored for retrieval tasks that require making discriminative distinctions between candidates. To address this, we propose $\textbf{CaRe-DPO}$, a retrieval framework that directly optimizes caption generation using retrieval relevance scores. At its core is Dual-Group Direct Preference Optimization (DG-DPO), a novel learning strategy that supervises captioning by modeling preferences across groups of distinct video and caption pairs. In addition, we present an MLLM-based retrieval model that incorporates role-embeddings to better distinguish between textual inputs with different functional roles, such as an auxiliary caption and a text query. Through extensive experiments, we demonstrate that CaRe-DPO significantly enhances retrieval performance by effectively leveraging auxiliary knowledge to generate fine-grained captions for retrieval. Code is available at this https URL.
Abstract (translated)
在文本-视频检索中，辅助字幕常被用来增强对视频的理解，从而弥合不同模态之间的差距。尽管最近多模态大型语言模型（MLLM）的发展已经实现了强大的零样本字幕生成能力，但我们观察到这种字幕通常缺乏特异性，在视觉相似的视频之间难以区分，因此在精细检索任务中实用性有限。此外，传统的字幕生成方法主要通过诸如BLEU等语言生成指标进行评估，这些指标并不特别适合需要对候选项做出区别性判断的检索任务。为解决这一问题，我们提出了$\textbf{CaRe-DPO}$，一种新的检索框架，它直接利用检索相关分数来优化字幕生成过程。该框架的核心是双组直接偏好优化（Dual-Group Direct Preference Optimization, DG-DPO），这是一种新颖的学习策略，通过模拟不同视频和字幕对之间的偏好差异来进行监督学习。此外，我们还提出了一种基于MLLM的检索模型，该模型引入角色嵌入以更好地区分具有不同功能的角色的文本输入，例如辅助字幕和文本查询。 通过广泛的实验，我们证明了CaRe-DPO能够显著提升检索性能，有效地利用辅助信息生成用于检索任务的细粒度字幕。代码可在上述网址获取。
URL
https://arxiv.org/abs/2509.16560
PDF
-
AutoArabic: A Three-Stage Framework for Localizing Video-Text Retrieval Benchmarks
2025-09-19 21:35:04Mohamed Eltahir, Osamah Sarraj, Abdulrahman Alfrihidi, Taha Alshatiri, Mohammed Khurd, Mohammed Bremoo, Tanveer HussainAbstract
Video-to-text and text-to-video retrieval are dominated by English benchmarks (e.g. DiDeMo, MSR-VTT) and recent multilingual corpora (e.g. RUDDER), yet Arabic remains underserved, lacking localized evaluation metrics. We introduce a three-stage framework, AutoArabic, utilizing state-of-the-art large language models (LLMs) to translate non-Arabic benchmarks into Modern Standard Arabic, reducing the manual revision required by nearly fourfold. The framework incorporates an error detection module that automatically flags potential translation errors with 97% accuracy. Applying the framework to DiDeMo, a video retrieval benchmark produces DiDeMo-AR, an Arabic variant with 40,144 fluent Arabic descriptions. An analysis of the translation errors is provided and organized into an insightful taxonomy to guide future Arabic localization efforts. We train a CLIP-style baseline with identical hyperparameters on the Arabic and English variants of the benchmark, finding a moderate performance gap (about 3 percentage points at Recall@1), indicating that Arabic localization preserves benchmark difficulty. We evaluate three post-editing budgets (zero/ flagged-only/ full) and find that performance improves monotonically with more post-editing, while the raw LLM output (zero-budget) remains usable. To ensure reproducibility to other languages, we made the code available at this https URL.
Abstract (translated)
视频到文本和文本到视频的检索任务目前主要依赖于英语基准（如DiDeMo、MSR-VTT）以及最近的多语言语料库（例如RUDDER），然而阿拉伯语却相对较少被关注，缺少本地化的评估指标。为此，我们提出了一种三阶段框架——AutoArabic，该框架利用最先进的大规模语言模型（LLMs）将非阿拉伯语基准翻译成现代标准阿拉伯语，从而减少了近四倍的手动修订工作量。该框架还包含一个错误检测模块，能够以97%的准确率自动标记潜在的翻译错误。 应用此框架对视频检索基准DiDeMo进行处理后，我们生成了名为DiDeMo-AR的阿拉伯语变体版本，其中包含了40,144个流利的阿拉伯文描述。我们还提供了一项关于翻译错误的分析，并将其组织成一个具有洞察力的分类法，以指导未来的阿拉伯语本地化工作。 我们在阿拉伯语和英语两种语言的基准上训练了一个CLIP风格的基础模型（使用相同的超参数），发现性能差距中等（Recall@1指标下约3个百分点），这表明阿拉伯语本地化保持了基准任务的难度不变。我们评估了三种后续编辑预算方案（零预算/仅标记错误/全面编辑）并发现，随着更多后续编辑工作的增加，性能会单调递增，而原始LLM输出（零预算版本）仍然可用。 为了确保该框架可以推广到其他语言的应用中，我们将代码开源至提供的网址。
URL
https://arxiv.org/abs/2509.16438
PDF
-
Hybrid-Tower: Fine-grained Pseudo-query Interaction and Generation for Text-to-Video Retrieval
2025-09-05 03:05:50Bangxiang Lan, Ruobing Xie, Ruixiang Zhao, Xingwu Sun, Zhanhui Kang, Gang Yang, Xirong LiAbstract
The Text-to-Video Retrieval (T2VR) task aims to retrieve unlabeled videos by textual queries with the same semantic meanings. Recent CLIP-based approaches have explored two frameworks: Two-Tower versus Single-Tower framework, yet the former suffers from low effectiveness, while the latter suffers from low efficiency. In this study, we explore a new Hybrid-Tower framework that can hybridize the advantages of the Two-Tower and Single-Tower framework, achieving high effectiveness and efficiency simultaneously. We propose a novel hybrid method, Fine-grained Pseudo-query Interaction and Generation for T2VR, ie, PIG, which includes a new pseudo-query generator designed to generate a pseudo-query for each video. This enables the video feature and the textual features of pseudo-query to interact in a fine-grained manner, similar to the Single-Tower approaches to hold high effectiveness, even before the real textual query is received. Simultaneously, our method introduces no additional storage or computational overhead compared to the Two-Tower framework during the inference stage, thus maintaining high efficiency. Extensive experiments on five commonly used text-video retrieval benchmarks demonstrate that our method achieves a significant improvement over the baseline, with an increase of $1.6\% \sim 3.9\%$ in R@1. Furthermore, our method matches the efficiency of Two-Tower models while achieving near state-of-the-art performance, highlighting the advantages of the Hybrid-Tower framework.
Abstract (translated)
文本到视频检索（T2VR）任务的目标是通过具有相同语义含义的文本查询来检索未标记的视频。最近基于CLIP的方法探索了两种框架：Two-Tower与Single-Tower框架，前者在有效性方面表现不佳，而后者在效率方面存在不足。在这项研究中，我们提出了一种新的Hybrid-Tower框架，该框架结合了Two-Tower和Single-Tower框架的优点，在保持高效率的同时实现高有效性。我们提出了一种新颖的混合方法——细粒度伪查询交互与生成（PIG），它包括一个新的伪查询生成器，旨在为每个视频生成一个伪查询。这使得视频特征与伪查询的文本特征能够以细粒度方式进行互动，类似于Single-Tower方法，在实际收到文本查询之前就能保持高有效性。同时，我们的方法在推理阶段不引入额外的存储或计算开销，从而保持了高效率。在五个常用的文字-视频检索基准上进行的广泛实验表明，与基线相比，我们的方法取得了显著改进，R@1提升了$1.6\% \sim 3.9\%$。此外，我们的方法达到了Two-Tower模型的效率水平，并实现了接近最先进性能的效果，突显了Hybrid-Tower框架的优势。
URL
https://arxiv.org/abs/2509.04773
PDF
-
Enhancing Partially Relevant Video Retrieval with Robust Alignment Learning
2025-09-01 11:30:43Long Zhang, Peipei Song, Jianfeng Dong, Kun Li, Xun YangAbstract
Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos partially relevant to a given query. The core challenge lies in learning robust query-video alignment against spurious semantic correlations arising from inherent data uncertainty: 1) query ambiguity, where the query incompletely characterizes the target video and often contains uninformative tokens, and 2) partial video relevance, where abundant query-irrelevant segments introduce contextual noise in cross-modal alignment. Existing methods often focus on enhancing multi-scale clip representations and retrieving the most relevant clip. However, the inherent data uncertainty in PRVR renders them vulnerable to distractor videos with spurious similarities, leading to suboptimal performance. To fill this research gap, we propose Robust Alignment Learning (RAL) framework, which explicitly models the uncertainty in data. Key innovations include: 1) we pioneer probabilistic modeling for PRVR by encoding videos and queries as multivariate Gaussian distributions. This not only quantifies data uncertainty but also enables proxy-level matching to capture the variability in cross-modal correspondences; 2) we consider the heterogeneous informativeness of query words and introduce learnable confidence gates to dynamically weight similarity. As a plug-and-play solution, RAL can be seamlessly integrated into the existing architectures. Extensive experiments across diverse retrieval backbones demonstrate its effectiveness.
Abstract (translated)
部分相关视频检索（PRVR）旨在根据给定的查询检索与之部分相关的未裁剪视频。其核心挑战在于学习稳健的查询-视频对齐，以应对由于数据不确定性而产生的虚假语义关联：1) 查询模糊性，在这种情况下，查询不完全描述目标视频，并且通常包含无信息词汇；2) 部分相关视频，即大量与查询无关的片段引入了跨模态对齐中的上下文噪声。现有的方法往往集中在增强多尺度剪辑表示和检索最相关的剪辑上。然而，在PRVR中固有的数据不确定性使得这些方法容易受到具有虚假相似性的干扰视频的影响，从而导致次优性能。 为填补这一研究空白，我们提出了稳健对齐学习（RAL）框架，该框架明确建模了数据中的不确定性。关键技术创新包括：1) 我们率先为PRVR使用概率模型，并将视频和查询编码为多元高斯分布。这不仅量化了数据的不确定性，还能够通过代理级别匹配捕捉跨模态对应关系的变化；2) 考虑到查询词汇的信息异质性，我们引入可学习的信心门以动态加权相似度。作为即插即用解决方案，RAL可以无缝地集成到现有架构中。在各种检索骨干网络上的广泛实验表明了其有效性。
URL
https://arxiv.org/abs/2509.01383
PDF
-
Multi-Focused Video Group Activities Hashing
2025-08-30 13:22:57Zhongmiao Qi, Yan Jiang, Bolin Zhang, Lijun Guo, Chong Wang, Qiangbo QianAbstract
With the explosive growth of video data in various complex scenarios, quickly retrieving group activities has become an urgent problem. However, many tasks can only retrieve videos focusing on an entire video, not the activity granularity. To solve this problem, we propose a new STVH (spatiotemporal interleaved video hashing) technique for the first time. Through a unified framework, the STVH simultaneously models individual object dynamics and group interactions, capturing the spatiotemporal evolution on both group visual features and positional features. Moreover, in real-life video retrieval scenarios, it may sometimes require activity features, while at other times, it may require visual features of objects. We then further propose a novel M-STVH (multi-focused spatiotemporal video hashing) as an enhanced version to handle this difficult task. The advanced method incorporates hierarchical feature integration through multi-focused representation learning, allowing the model to jointly focus on activity semantics features and object visual features. We conducted comparative experiments on publicly available datasets, and both STVH and M-STVH can achieve excellent results.
Abstract (translated)
随着各种复杂场景中视频数据的爆炸性增长，快速检索群体活动已成为一个紧迫的问题。然而，许多任务只能针对整个视频进行检索，而不能达到活动粒度级别。为了解决这个问题，我们首次提出了一种新的时空交错视频哈希（STVH）技术。通过统一框架，STVH同时建模了个体对象的动态变化和群体互动，捕捉到了群体视觉特征以及位置特征上的时空演变。 然而，在实际的视频检索场景中，有时需要活动特征，而其他时候则可能需要物体的视觉特征。为此，我们进一步提出了一种新颖的多焦点时空视频哈希（M-STVH）作为增强版本来处理这一难题。该高级方法通过多层次表示学习整合了分层特性集成，使模型能够同时关注活动语义特性和对象视觉特性。 我们在公开数据集上进行了比较实验，结果显示STVH和M-STVH都能取得卓越的结果。
URL
https://arxiv.org/abs/2509.00490
PDF
-
ProPy: Building Interactive Prompt Pyramids upon CLIP for Partially Relevant Video Retrieval
2025-08-26 13:42:48Yi Pan, Yujia Zhang, Michael Kampffmeyer, Xiaoguang ZhaoAbstract
Partially Relevant Video Retrieval (PRVR) is a practical yet challenging task that involves retrieving videos based on queries relevant to only specific segments. While existing works follow the paradigm of developing models to process unimodal features, powerful pretrained vision-language models like CLIP remain underexplored in this field. To bridge this gap, we propose ProPy, a model with systematic architectural adaption of CLIP specifically designed for PRVR. Drawing insights from the semantic relevance of multi-granularity events, ProPy introduces two key innovations: (1) A Prompt Pyramid structure that organizes event prompts to capture semantics at multiple granularity levels, and (2) An Ancestor-Descendant Interaction Mechanism built on the pyramid that enables dynamic semantic interaction among events. With these designs, ProPy achieves SOTA performance on three public datasets, outperforming previous models by significant margins. Code is available at this https URL.
Abstract (translated)
部分相关视频检索（PRVR）是一项实用且具有挑战性的任务，涉及根据与特定片段相关的查询来检索视频。尽管现有的研究遵循开发用于处理单一模态特征的模型的方法，但强大的预训练视觉-语言模型如CLIP在这一领域中尚未得到充分探索。为了弥合这一差距，我们提出了ProPy，这是一种专门针对PRVR设计的、对CLIP进行了系统架构适应的模型。从多粒度事件语义相关性的见解出发，ProPy引入了两个关键创新：(1) 一个提示金字塔结构，用于组织事件提示以捕捉多个粒度级别的语义；(2) 基于该金字塔构建的祖裔交互机制，使事件之间能够进行动态的语义互动。凭借这些设计，ProPy在三个公共数据集上取得了SOTA（最先进的）性能，并且显著优于之前的模型。代码可在提供的链接中获取。
URL
https://arxiv.org/abs/2508.19024
PDF
-
Repeating Words for Video-Language Retrieval with Coarse-to-Fine Objectives
2025-08-20 16:03:56Haoyu Zhao, Jiaxi Gu, Shicong Wang, Xing Zhang, Hang Xu, Zuxuan Wu, Yu-Gang JiangAbstract
The explosive growth of video streaming presents challenges in achieving high accuracy and low training costs for video-language retrieval. However, existing methods rely on large-scale pre-training to improve video retrieval performance, resulting in significant computational demands. Additionally, the fine-grained information in videos and texts remains underexplored. To alleviate these problems, we propose a novel framework to learn fine-grained features for better alignment and introduce an inference pipeline to improve performance without additional training. Specifically, we employ coarse-to-fine objectives to understand the semantic information of video-text pairs, including contrastive and matching learning. The fine-grained data used for training is obtained through the Granularity-Aware Representation module, which is designed based on similarity analysis between video frames and words in captions. Furthermore, we observe that the repetition of keywords in the original captions, referred to as "Repetition", can enhance retrieval performance and improve alignment between video and text. Based on this insight, we propose a novel and effective inference pipeline that incorporates a voting mechanism and a new Matching Entropy metric to achieve better retrieval performance without requiring additional pre-training. Experimental results on four benchmarks demonstrate that the proposed method outperforms previous approaches. Additionally, our inference pipeline achieves significant performance improvements, with a 2.1% increase in Recall@1 on the MSR-VTT dataset and a 1.6% increase on the DiDeMo dataset.
Abstract (translated)
视频流媒体的爆炸性增长带来了在实现高精度和低成本训练的同时进行视频-语言检索的挑战。然而，现有的方法依赖大规模预训练来提升视频检索性能，导致了巨大的计算需求。此外，视频和文本中的细粒度信息仍未得到充分探索。为了解决这些问题，我们提出了一种新的框架，用于学习更精细的特征以实现更好的对齐，并引入了一个推理管道，在不增加额外训练的情况下提高性能。 具体来说，我们采用了从粗到细的目标来理解视频-文本对的语义信息，包括对比学习和匹配学习。通过基于视频帧与字幕中单词之间的相似性分析设计的Granularity-Aware Representation模块获得了用于训练的细粒度数据。此外，我们观察到在原始字幕中重复关键词（称为“重复”）可以提升检索性能，并增强视频与文本之间的一致性。基于这一发现，我们提出了一种新的且有效的推理管道，该管道整合了投票机制和一种新的匹配熵指标，在无需额外预训练的情况下实现更好的检索性能。 实验结果表明，所提方法在四个基准测试数据集上超越了先前的方法。此外，我们的推理管道取得了显著的性能提升：在MSR-VTT数据集上，Recall@1提高了2.1%，而在DiDeMo数据集上的提高为1.6%。
URL
https://arxiv.org/abs/2508.14812
PDF
-
Beyond Simple Edits: Composed Video Retrieval with Dense Modifications
2025-08-19 17:59:39Omkar Thawakar, Dmitry Demidov, Ritesh Thawkar, Rao Muhammad Anwer, Mubarak Shah, Fahad Shahbaz Khan, Salman KhanAbstract
Composed video retrieval is a challenging task that strives to retrieve a target video based on a query video and a textual description detailing specific modifications. Standard retrieval frameworks typically struggle to handle the complexity of fine-grained compositional queries and variations in temporal understanding limiting their retrieval ability in the fine-grained setting. To address this issue, we introduce a novel dataset that captures both fine-grained and composed actions across diverse video segments, enabling more detailed compositional changes in retrieved video content. The proposed dataset, named Dense-WebVid-CoVR, consists of 1.6 million samples with dense modification text that is around seven times more than its existing counterpart. We further develop a new model that integrates visual and textual information through Cross-Attention (CA) fusion using grounded text encoder, enabling precise alignment between dense query modifications and target videos. The proposed model achieves state-of-the-art results surpassing existing methods on all metrics. Notably, it achieves 71.3\% Recall@1 in visual+text setting and outperforms the state-of-the-art by 3.4\%, highlighting its efficacy in terms of leveraging detailed video descriptions and dense modification texts. Our proposed dataset, code, and model are available at :this https URL
Abstract (translated)
基于视频组成的检索是一项具有挑战性的任务，旨在根据查询视频和详细描述特定修改的文本描述来检索目标视频。标准的检索框架通常难以处理细粒度组合查询的复杂性和时间理解中的变化，这限制了它们在细粒度设置下的检索能力。为解决这一问题，我们引入了一个新颖的数据集，该数据集捕捉了多样视频片段中细粒度和组成动作的变化，从而使得检索到的视频内容能够进行更详细的组合更改。提出的名为Dense-WebVid-CoVR的数据集包含160万个样本，并且其密集修改文本量是现有同类数据集的七倍左右。 我们进一步开发了一种新模型，通过使用基于接地文本编码器的跨注意力（Cross-Attention, CA）融合方法来整合视觉和文本信息，从而能够精确地对齐密集查询修改与目标视频。所提出的模型在所有度量标准上均取得了超越现有方法的最佳成果。尤其值得注意的是，在视觉+文本设置下，它实现了71.3%的Recall@1，并比最先进的方法高出3.4%，显示了其利用详细视频描述和密集修改文本的有效性。 我们提出的数据集、代码以及模型可在以下链接获得：[提供URL]
URL
https://arxiv.org/abs/2508.14039
PDF
-
VimoRAG: Video-based Retrieval-augmented 3D Motion Generation for Motion Language Models
2025-08-16 15:31:14Haidong Xu, Guangwei Xu, Zhedong Zheng, Xiatian Zhu, Wei Ji, Xiangtai Li, Ruijie Guo, Meishan Zhang, Min zhang, Hao FeiAbstract
This paper introduces VimoRAG, a novel video-based retrieval-augmented motion generation framework for motion large language models (LLMs). As motion LLMs face severe out-of-domain/out-of-vocabulary issues due to limited annotated data, VimoRAG leverages large-scale in-the-wild video databases to enhance 3D motion generation by retrieving relevant 2D human motion signals. While video-based motion RAG is nontrivial, we address two key bottlenecks: (1) developing an effective motion-centered video retrieval model that distinguishes human poses and actions, and (2) mitigating the issue of error propagation caused by suboptimal retrieval results. We design the Gemini Motion Video Retriever mechanism and the Motion-centric Dual-alignment DPO Trainer, enabling effective retrieval and generation processes. Experimental results show that VimoRAG significantly boosts the performance of motion LLMs constrained to text-only input.
Abstract (translated)
本文介绍了VimoRAG，这是一种新型的基于视频检索增强的动作生成框架，专门用于运动大型语言模型（LLMs）。由于受限于有限的标注数据，运动LLMs面临着严重的域外/词汇外问题。为此，VimoRAG利用大规模的真实世界视频数据库来通过检索相关的2D人体动作信号提升3D运动生成能力。虽然基于视频的动作检索增强（motion RAG）具有挑战性，我们解决了两个关键瓶颈：(1) 开发一个有效的以动作为中心的视频检索模型，该模型能够区分人类的姿态和行为；以及 (2) 减轻由于次优检索结果导致的误差传播问题。为此，我们设计了双子星运动视频检索器（Gemini Motion Video Retriever）机制和以运动为中心的双重对齐DPO训练器（Motion-centric Dual-alignment DPO Trainer），使有效的检索与生成过程得以实现。实验结果显示，VimoRAG显著提升了仅基于文本输入约束下的运动LLMs的性能。
URL
https://arxiv.org/abs/2508.12081
PDF
-
TrajSV: A Trajectory-based Model for Sports Video Representations and Applications
2025-08-15 16:23:36Zheng Wang, Shihao Xu, Wei ShiAbstract
Sports analytics has received significant attention from both academia and industry in recent years. Despite the growing interest and efforts in this field, several issues remain unresolved, including (1) data unavailability, (2) lack of an effective trajectory-based framework, and (3) requirement for sufficient supervision labels. In this paper, we present TrajSV, a trajectory-based framework that addresses various issues in existing studies. TrajSV comprises three components: data preprocessing, Clip Representation Network (CRNet), and Video Representation Network (VRNet). The data preprocessing module extracts player and ball trajectories from sports broadcast videos. CRNet utilizes a trajectory-enhanced Transformer module to learn clip representations based on these trajectories. Additionally, VRNet learns video representations by aggregating clip representations and visual features with an encoder-decoder architecture. Finally, a triple contrastive loss is introduced to optimize both video and clip representations in an unsupervised manner. The experiments are conducted on three broadcast video datasets to verify the effectiveness of TrajSV for three types of sports (i.e., soccer, basketball, and volleyball) with three downstream applications (i.e., sports video retrieval, action spotting, and video captioning). The results demonstrate that TrajSV achieves state-of-the-art performance in sports video retrieval, showcasing a nearly 70% improvement. It outperforms baselines in action spotting, achieving state-of-the-art results in 9 out of 17 action categories, and demonstrates a nearly 20% improvement in video captioning. Additionally, we introduce a deployed system along with the three applications based on TrajSV.
Abstract (translated)
近年来，体育数据分析领域受到了学术界和工业界的广泛关注。尽管该领域的兴趣与努力日益增加，但仍存在一些未解决的问题，包括（1）数据不可用性，（2）缺乏有效的基于轨迹的框架，以及（3）需要足够的监督标签。在本文中，我们提出了TrajSV，这是一种基于轨迹的框架，旨在解决现有研究中存在的各种问题。TrajSV包含三个组成部分：数据预处理模块、片段表示网络（CRNet）和视频表示网络（VRNet）。数据预处理模块从体育广播视频中提取球员和球的运动轨迹。CRNet利用增强型Transformer模块根据这些轨迹学习片段表示。此外，VRNet通过编码器-解码器架构聚合片段表示和视觉特征来学习视频表示。最后，引入了三重对比损失以在无监督的方式下优化视频和片段表示。我们在三个广播视频数据集上进行了实验，以验证TrajSV对于三种体育项目（即足球、篮球和排球）的三项下游应用（即体育视频检索、动作识别和视频字幕生成）的有效性。结果表明，TrajSV在体育视频检索方面取得了最先进的性能，并且比基线方法提高了近70%；它在9个类别中的17个类别的动作识别中获得了最佳效果；并且在视频字幕生成上几乎提高了20%的性能。此外，我们还介绍了一个基于TrajSV的部署系统以及上述三个应用。
URL
https://arxiv.org/abs/2508.11569
PDF
-
Adversarial Video Promotion Against Text-to-Video Retrieval
2025-08-09 12:20:13Qiwei Tian, Chenhao Lin, Zhengyu Zhao, Qian Li, Shuai Liu, Chao ShenAbstract
Thanks to the development of cross-modal models, text-to-video retrieval (T2VR) is advancing rapidly, but its robustness remains largely unexamined. Existing attacks against T2VR are designed to push videos away from queries, i.e., suppressing the ranks of videos, while the attacks that pull videos towards selected queries, i.e., promoting the ranks of videos, remain largely unexplored. These attacks can be more impactful as attackers may gain more views/clicks for financial benefits and widespread (mis)information. To this end, we pioneer the first attack against T2VR to promote videos adversarially, dubbed the Video Promotion attack (ViPro). We further propose Modal Refinement (MoRe) to capture the finer-grained, intricate interaction between visual and textual modalities to enhance black-box transferability. Comprehensive experiments cover 2 existing baselines, 3 leading T2VR models, 3 prevailing datasets with over 10k videos, evaluated under 3 scenarios. All experiments are conducted in a multi-target setting to reflect realistic scenarios where attackers seek to promote the video regarding multiple queries simultaneously. We also evaluated our attacks for defences and imperceptibility. Overall, ViPro surpasses other baselines by over $30/10/4\%$ for white/grey/black-box settings on average. Our work highlights an overlooked vulnerability, provides a qualitative analysis on the upper/lower bound of our attacks, and offers insights into potential counterplays. Code will be publicly available at this https URL.
Abstract (translated)
感谢跨模态模型的发展，文本到视频检索（T2VR）正在迅速进步，但其鲁棒性仍然很大程度上未被研究。现有针对T2VR的攻击旨在将视频从查询中推开，即降低视频排名；而能够将视频拉向选定查询、即提升视频排名的攻击则尚未充分探索。此类攻击可能更具影响力，因为攻击者可能会为了经济利益和广泛传播（错误）信息而获得更多点击/观看。为此，我们开创了首个旨在通过敌对方式推广视频的T2VR攻击，称为“Video Promotion”攻击（简称ViPro）。此外，我们提出了模态细化（Modal Refinement, MoRe），以捕捉视觉与文本模态之间更为细微复杂的交互作用，从而增强黑盒转移能力。全面的实验涵盖了2个现有基线、3种领先的T2VR模型以及包含超过10k视频的3大流行数据集，在3种不同场景下进行了评估。所有实验均在多目标设置中进行，以反映现实情况中的攻击者试图同时针对多个查询推广视频的情景。我们还对防御措施和不可察觉性进行了测试。总体而言，ViPro在白盒/灰盒/黑盒环境下分别平均超过了其他基线30%/10%/4%以上。我们的工作揭示了一个被忽视的漏洞，提供了对我们攻击上下限的定性分析，并为潜在的反击策略提供见解。代码将在该链接公开发布。
URL
https://arxiv.org/abs/2508.06964
PDF
-
GAID: Frame-Level Gated Audio-Visual Integration with Directional Perturbation for Text-Video Retrieval
2025-08-03 10:44:24Bowen Yang, Yun Cao, Chen He, Xiaosu SuAbstract
Text-to-video retrieval requires precise alignment between language and temporally rich video signals. Existing methods predominantly exploit visual cues and often overlook complementary audio semantics or adopt coarse fusion strategies, leading to suboptimal multimodal representations. We present GAID, a framework that jointly address this gap via two key components: (i) a Frame-level Gated Fusion (FGF) that adaptively integrates audio and visual features under textual guidance, enabling fine-grained temporal alignment; and (ii) a Directional Adaptive Semantic Perturbation (DASP) that injects structure-aware perturbations into text embeddings, enhancing robustness and discrimination without incurring multi-pass inference. These modules complement each other -- fusion reduces modality gaps while perturbation regularizes cross-modal matching -- yielding more stable and expressive representations. Extensive experiments on MSR-VTT, DiDeMo, LSMDC, and VATEX show consistent state-of-the-art results across all retrieval metrics with notable efficiency gains. Our code is available at this https URL.
Abstract (translated)
文本到视频检索要求语言与时间丰富的视频信号之间精确对齐。现有方法主要利用视觉线索，并常常忽视互补的音频语义或采用粗略的融合策略，导致多模态表示次优。我们提出了GAID框架，通过两个关键组件来共同解决这一问题：(i) 帧级门控融合（FGF），在文本引导下自适应地整合音频和视觉特征，从而实现精细的时间对齐；以及(ii) 方向性自适应语义扰动（DASP），将结构感知的扰动注入到文本嵌入中，增强鲁棒性和区分度而不产生多遍推理。这些模块相互补充——融合减少模态差距而扰动则规范跨模态匹配，从而生成更稳定和表达性的表示。在MSR-VTT、DiDeMo、LSMDC 和 VATEX 数据集上的广泛实验显示，在所有检索指标上均取得了一致的最优结果，并且效率显著提高。我们的代码可在提供的链接获取。
URL
https://arxiv.org/abs/2508.01711
PDF
-
Bidirectional Likelihood Estimation with Multi-Modal Large Language Models for Text-Video Retrieval
2025-07-31 06:57:28Dohwan Ko, Ji Soo Lee, Minhyuk Choi, Zihang Meng, Hyunwoo J. KimAbstract
Text-Video Retrieval aims to find the most relevant text (or video) candidate given a video (or text) query from large-scale online databases. Recent work leverages multi-modal large language models (MLLMs) to improve retrieval, especially for long or complex query-candidate pairs. However, we observe that the naive application of MLLMs, i.e., retrieval based on candidate likelihood, introduces candidate prior bias, favoring candidates with inherently higher priors over those more relevant to the query. To this end, we propose a novel retrieval framework, Bidirectional Likelihood Estimation with MLLM (BLiM), which leverages both query and candidate likelihoods by training the model to generate text from a given video as well as video features from a given text. Furthermore, we introduce Candidate Prior Normalization (CPN), a simple yet effective training-free score calibration module designed to mitigate candidate prior bias in candidate likelihood. On four Text-Video Retrieval benchmarks, our BLiM equipped with CPN outperforms previous state-of-the-art models by 6.4 R@1 on average, effectively alleviating candidate prior bias and emphasizing query-candidate relevance. Our in-depth analysis across various multi-modal tasks beyond retrieval highlights the broad applicability of CPN which enhances visual understanding by reducing reliance on textual priors. Code is available at this https URL.
Abstract (translated)
文本-视频检索的目标是从大规模在线数据库中，根据给定的视频（或文本）查询找到最相关的文本（或视频）。最近的研究利用了多模态大型语言模型（MLLMs）来改进检索性能，特别是对于长或者复杂的查询候选对。然而，我们观察到直接应用MLLMs，即基于候选可能性进行检索的方法，引入了候选先验偏见，倾向于选择那些本身具有更高先验值的候选对象而非与查询更相关的对象。 为此，我们提出了一个新颖的检索框架——双向似然估计多模态模型（BLiM），该框架利用了查询和候选的概率，通过训练模型从给定视频生成文本以及从给定文本生成视频特征。此外，我们引入了一种简单而有效的无需训练的评分校准模块——候选先验归一化（CPN），专门用于缓解候选概率中的候选先验偏见问题。 在四个文本-视频检索基准测试中，配备有CPN的BLiM模型平均提高了6.4个R@1点的性能，有效减轻了候选先验偏见，并强调了查询-候选的相关性。我们对各种多模态任务（超出检索范围）进行了深入分析，表明CPN具有广泛的应用潜力，通过减少对文本先验的依赖来增强视觉理解能力。 代码可在[此处](https://this.is.a.mock.url)获取。（注：实际链接应替换为真实提供的URL地址。）
URL
https://arxiv.org/abs/2507.23284
PDF
-
HLFormer: Enhancing Partially Relevant Video Retrieval with Hyperbolic Learning
2025-07-23 10:59:46Li Jun, Wang Jinpeng, Tan Chaolei, Lian Niu, Chen Long, Zhang Min, Wang Yaowei, Xia Shu-Tao, Chen BinAbstract
Partially Relevant Video Retrieval (PRVR) addresses the critical challenge of matching untrimmed videos with text queries describing only partial content. Existing methods suffer from geometric distortion in Euclidean space that sometimes misrepresents the intrinsic hierarchical structure of videos and overlooks certain hierarchical semantics, ultimately leading to suboptimal temporal modeling. To address this issue, we propose the first hyperbolic modeling framework for PRVR, namely HLFormer, which leverages hyperbolic space learning to compensate for the suboptimal hierarchical modeling capabilities of Euclidean space. Specifically, HLFormer integrates the Lorentz Attention Block and Euclidean Attention Block to encode video embeddings in hybrid spaces, using the Mean-Guided Adaptive Interaction Module to dynamically fuse features. Additionally, we introduce a Partial Order Preservation Loss to enforce "text < video" hierarchy through Lorentzian cone constraints. This approach further enhances cross-modal matching by reinforcing partial relevance between video content and text queries. Extensive experiments show that HLFormer outperforms state-of-the-art methods. Code is released at this https URL.
Abstract (translated)
部分相关视频检索（PRVR）旨在解决未裁剪视频与仅描述部分内容的文本查询之间的匹配难题。现有方法在欧几里得空间中存在几何失真，这有时会错误地表示视频内在层次结构，并忽略某些层级语义，最终导致次优的时间建模效果。为了解决这一问题，我们提出了首个用于PRVR的双曲线建模框架HLFormer，该框架利用双曲空间学习来弥补欧几里得空间在层级建模能力上的不足。 具体来说，HLFormer结合了洛伦兹注意力块和欧氏注意力块，在混合空间中编码视频嵌入，并使用均值引导自适应交互模块动态融合特征。此外，我们引入了一个部分顺序保持损失函数，通过洛伦茨锥约束来强制执行“文本<视频”的层级关系。这种方法通过强化视频内容与文本查询之间的部分相关性进一步增强了跨模态匹配。 广泛的实验表明，HLFormer优于当前最先进的方法。代码可在以下网址获取：[此URL]（请替换为实际的URL）。
URL
https://arxiv.org/abs/2507.17402
PDF
-
Quantifying and Narrowing the Unknown: Interactive Text-to-Video Retrieval via Uncertainty Minimization
2025-07-21 11:12:39Bingqing Zhang, Zhuo Cao, Heming Du, Yang Li, Xue Li, Jiajun Liu, Sen WangAbstract
Despite recent advances, Text-to-video retrieval (TVR) is still hindered by multiple inherent uncertainties, such as ambiguous textual queries, indistinct text-video mappings, and low-quality video frames. Although interactive systems have emerged to address these challenges by refining user intent through clarifying questions, current methods typically rely on heuristic or ad-hoc strategies without explicitly quantifying these uncertainties, limiting their effectiveness. Motivated by this gap, we propose UMIVR, an Uncertainty-Minimizing Interactive Text-to-Video Retrieval framework that explicitly quantifies three critical uncertainties-text ambiguity, mapping uncertainty, and frame uncertainty-via principled, training-free metrics: semantic entropy-based Text Ambiguity Score (TAS), Jensen-Shannon divergence-based Mapping Uncertainty Score (MUS), and a Temporal Quality-based Frame Sampler (TQFS). By adaptively generating targeted clarifying questions guided by these uncertainty measures, UMIVR iteratively refines user queries, significantly reducing retrieval ambiguity. Extensive experiments on multiple benchmarks validate UMIVR's effectiveness, achieving notable gains in Recall@1 (69.2\% after 10 interactive rounds) on the MSR-VTT-1k dataset, thereby establishing an uncertainty-minimizing foundation for interactive TVR.
Abstract (translated)
尽管最近取得了进展，文本到视频检索（TVR）仍然受到多种内在不确定性的阻碍，例如模棱两可的文本查询、不明确的文本与视频映射关系以及低质量的视频帧。虽然交互式系统已经出现，通过提出澄清问题来细化用户意图以应对这些挑战，但当前的方法通常依赖于启发式或临时策略，并未显式量化这些不确定性，从而限制了其有效性。为了解决这一差距，我们提出了UMIVR（不确定性最小化的交互式文本到视频检索框架），该框架通过基于原则的、无训练的数据方法明确地量化三种关键不确定性：语义熵为基础的文本歧义评分（TAS）、基于杰森-香农散度的映射不确定评分（MUS）和基于时间质量的时间帧采样器（TQFS）。UMIVR能够根据这些不确定性衡量标准自适应生成有针对性的澄清问题，从而迭代细化用户查询，并显著减少检索模糊性。在多个基准测试中的广泛实验验证了UMIVR的有效性，在MSR-VTT-1k数据集上实现了Recall@1在10轮交互后达到69.2%的良好效果，为交互式TVR建立了基于最小化不确定性的基础。
URL
https://arxiv.org/abs/2507.15504
PDF
