Abstract
Text-to-Video (T2V) retrieval aims to identify the most relevant item from a gallery of videos based on a user's text query. Traditional methods rely solely on aligning video and text modalities to compute the similarity and retrieve relevant items. However, recent advancements emphasise incorporating auxiliary information extracted from video and text modalities to improve retrieval performance and bridge the semantic gap between these modalities. Auxiliary information can include visual attributes, such as objects; temporal and spatial context; and textual descriptions, such as speech and rephrased captions. This survey comprehensively reviews 81 research papers on Text-to-Video retrieval that utilise such auxiliary information. It provides a detailed analysis of their methodologies; highlights state-of-the-art results on benchmark datasets; and discusses available datasets and their auxiliary information. Additionally, it proposes promising directions for future research, focusing on different ways to further enhance retrieval performance using this information.
Abstract (translated)
文本到视频（T2V）检索的目标是根据用户的文本查询从一系列视频中识别出最相关的视频。传统的方法主要依赖于对齐视频和文本模态以计算相似性并检索相关项。然而，最近的研究强调了利用从视频和文本模态中提取的辅助信息来改善检索性能，并弥合这些模态之间的语义差距的重要性。这种辅助信息可以包括视觉属性（如对象）、时间和空间上下文以及文本描述（如对话和重写的字幕）。本综述全面回顾了81篇关于利用此类辅助信息进行文本到视频检索的研究论文，详细分析了它们的方法论；强调了在基准数据集上的最新成果，并讨论了可用的数据集及其辅助信息。此外，它还提出了未来研究的有前景的方向，重点是通过使用这些信息来进一步提高检索性能的不同方法。
URL
https://arxiv.org/abs/2505.23952
