Abstract
This report presents our solution to the Ego4D Natural Language Queries (NLQ) Challenge at CVPR 2025. Egocentric video captures the scene from the wearer's perspective, where gaze serves as a key non-verbal communication cue that reflects visual attention and offer insights into human intention and cognition. Motivated by this, we propose a novel approach, GazeNLQ, which leverages gaze to retrieve video segments that match given natural language queries. Specifically, we introduce a contrastive learning-based pretraining strategy for gaze estimation directly from video. The estimated gaze is used to augment video representations within proposed model, thereby enhancing localization accuracy. Experimental results show that GazeNLQ achieves R1@IoU0.3 and R1@IoU0.5 scores of 27.82 and 18.68, respectively. Our code is available at this https URL.
Abstract (translated)
该报告介绍了我们对CVPR 2025年Ego4D自然语言查询（NLQ）挑战赛的解决方案。自拍视频从穿戴者的视角捕捉场景，其中视线是关键的非言语交流线索，反映了视觉注意力，并提供了有关人类意图和认知的见解。受此启发，我们提出了一种新颖的方法GazeNLQ，该方法利用视线来检索与给定自然语言查询相匹配的视频片段。具体而言，我们引入了一种基于对比学习的预训练策略，直接从视频中进行视线估计。估算出的视线被用来增强模型中的视频表示，从而提高定位精度。实验结果显示，GazeNLQ在R1@IoU0.3和R1@IoU0.5评分上分别取得了27.82和18.68的成绩。我们的代码可在[此处](https://this_https_URL.com)获取。
URL
https://arxiv.org/abs/2506.05782
