Abstract
Recent advances in text-video retrieval have been largely driven by contrastive learning frameworks. However, existing methods overlook a key source of optimization tension: the separation between text and video distributions in the representation space (referred to as the modality gap), and the prevalence of false negatives in batch sampling. These factors lead to conflicting gradients under the InfoNCE loss, impeding stable alignment. To mitigate this, we propose GARE, a Gap-Aware Retrieval framework that introduces a learnable, pair-specific increment Delta_ij between text t_i and video v_j to offload the tension from the global anchor representation. We first derive the ideal form of Delta_ij via a coupled multivariate first-order Taylor approximation of the InfoNCE loss under a trust-region constraint, revealing it as a mechanism for resolving gradient conflicts by guiding updates along a locally optimal descent direction. Due to the high cost of directly computing Delta_ij, we introduce a lightweight neural module conditioned on the semantic gap between each video-text pair, enabling structure-aware correction guided by gradient supervision. To further stabilize learning and promote interpretability, we regularize Delta using three components: a trust-region constraint to prevent oscillation, a directional diversity term to promote semantic coverage, and an information bottleneck to limit redundancy. Experiments across four retrieval benchmarks show that GARE consistently improves alignment accuracy and robustness to noisy supervision, confirming the effectiveness of gap-aware tension mitigation.
Abstract (translated)
最近在文本-视频检索领域的进展主要受到对比学习框架的推动。然而，现有的方法忽视了一个重要的优化张力来源：即文本和视频在表示空间中的分布分离（称为模态差距）以及批量采样中假阴性的普遍存在。这些因素导致了在InfoNCE损失下的梯度冲突，阻碍了稳定的对齐过程。 为了解决这个问题，我们提出了GARE（Gap-Aware Retrieval），这是一种基于感知间隙的检索框架，它引入了一个可学习的、特定于每对的增量Delta_ij，在文本t_i和视频v_j之间。通过这种方式，它可以将张力从全局锚表示卸载下来。首先，我们在受信任区域约束的情况下通过耦合多元一阶泰勒近似导出了Delta_ij的理想形式，揭示了它作为解决梯度冲突机制的作用，引导沿着局部最优下降方向的更新。 由于直接计算Delta_ij的成本很高，我们引入了一个轻量级神经模块，该模块基于每对视频-文本之间的语义差距进行条件化处理。这使得能够通过梯度监督来进行结构感知校正。为了进一步稳定学习并促进可解释性，我们使用三个部分来规范Delta：一个信任区域约束以防止振荡，一个方向多样性项以促进语义覆盖，并一个信息瓶颈以限制冗余。 在四个检索基准上的实验表明，GARE能够一致地提高对齐准确性和对噪声监督的鲁棒性，证实了感知间隙张力缓解的有效性。
URL
https://arxiv.org/abs/2505.12499
