Abstract
Enabling efficient text-video retrieval on edge-end devices is critical for real-world applications. Yet, existing methods face a critical challenge in balancing accuracy and computational efficiency: uniform frame sampling methods ensure content coverage but incur prohibitive computational costs, while salient-frame sampling methods reduce overhead but suffer from query-agnostic frame selection that biases retrieval results. To address this, we propose ProCLIP, a user-centric framework that achieves state-of-the-art accuracy with significantly improved efficiency. We design a prompt-aware frame sampling strategy that dynamically guides lightweight feature extractors using textual prompts to select semantically relevant frames, overcoming the limitations of existing salient-frame sampling methods which rely on static, query-agnostic selection criteria. Moreover, we adopt a two-stage candidate pruning strategy that combines rapid coarse filtering via a lightweight module with CLIP-powered fine-grained re-ranking, enhancing retrieval efficiency while preserving accuracy. Experiments across benchmarks show ProCLIP achieves 75.3% latency reduction versus baselines while maintaining competitive accuracy, i.e., R@1=49.0 in MSR-VTT dataset. Code is available at this https URL.
Abstract (translated)
在边缘设备上实现高效的文字-视频检索对于实际应用至关重要。然而，现有的方法面临着如何平衡准确性和计算效率的关键挑战：统一的帧采样方法虽然确保了内容覆盖范围，但带来了高昂的计算成本；而显著帧采样方法则通过减少开销来缓解这一问题，但由于缺乏基于查询的相关性选择标准，它们可能导致检索结果偏向。为解决这些问题，我们提出了ProCLIP，这是一种以用户为中心的框架，能够在保持极高准确性的前提下大幅提升效率。 我们的框架设计了一个提示感知的帧采样策略，该策略动态地使用文本提示引导轻量级特征提取器选取语义相关的帧，从而克服了现有显著帧采样方法依赖于静态且与查询无关的选择标准这一局限性。此外，我们还采用了一种两阶段候选筛选策略，结合轻量化模块的快速粗略过滤和基于CLIP（Contrastive Language-Image Pre-training）技术的精细重排名，以此提高检索效率并保持准确性。 实验结果表明，在多个基准测试中，ProCLIP相较于基线方法减少了75.3%的延迟时间，并在MSR-VTT数据集上依然保持了49.0%的准确率（以R@1度量）。该框架的代码可在提供的链接获取。
URL
https://arxiv.org/abs/2507.15491
