Abstract
Partially Relevant Video Retrieval (PRVR) is a challenging task in the domain of multimedia retrieval. It is designed to identify and retrieve untrimmed videos that are partially relevant to the provided query. In this work, we investigate long-sequence video content understanding to address information redundancy issues. Leveraging the outstanding long-term state space modeling capability and linear scalability of the Mamba module, we introduce a multi-Mamba module with temporal fusion framework (MamFusion) tailored for PRVR task. This framework effectively captures the state-relatedness in long-term video content and seamlessly integrates it into text-video relevance understanding, thereby enhancing the retrieval process. Specifically, we introduce Temporal T-to-V Fusion and Temporal V-to-T Fusion to explicitly model temporal relationships between text queries and video moments, improving contextual awareness and retrieval accuracy. Extensive experiments conducted on large-scale datasets demonstrate that MamFusion achieves state-of-the-art performance in retrieval effectiveness. Code is available at the link: this https URL.
Abstract (translated)
部分相关视频检索（PRVR）是多媒体检索领域中的一个挑战性任务，旨在识别和提取与提供的查询部分相关的未剪辑视频。在这项工作中，我们研究了长序列视频内容理解以解决信息冗余问题。通过利用Mamba模块出色的长期状态空间建模能力和线性扩展能力，我们引入了一个配备时间融合框架的多Mamba模块（MamFusion），专门用于PRVR任务。该框架能够有效捕捉长期视频内容中的状态相关性，并将其无缝集成到文本与视频的相关理解中，从而提升检索过程的效果。 具体来说，我们介绍了时间T-to-V融合和时间V-to-T融合，以明确建模文本查询和视频时刻之间的时间关系，提高上下文感知能力和检索准确性。在大规模数据集上进行的大量实验表明，MamFusion在检索效果方面达到了最先进的性能。代码可在以下链接获取：this https URL.
URL
https://arxiv.org/abs/2506.03473
