Abstract
Unconstrained gaze estimation is the process of determining where a subject is directing their visual attention in uncontrolled environments. Gaze estimation systems are important for a myriad of tasks such as driver distraction monitoring, exam proctoring, accessibility features in modern software, etc. However, these systems face challenges in real-world scenarios, partially due to the low resolution of in-the-wild images and partially due to insufficient modeling of head-eye interactions in current state-of-the-art (SOTA) methods. This paper introduces DHECA-SuperGaze, a deep learning-based method that advances gaze prediction through super-resolution (SR) and a dual head-eye cross-attention (DHECA) module. Our dual-branch convolutional backbone processes eye and multiscale SR head images, while the proposed DHECA module enables bidirectional feature refinement between the extracted visual features through cross-attention mechanisms. Furthermore, we identified critical annotation errors in one of the most diverse and widely used gaze estimation datasets, Gaze360, and rectified the mislabeled data. Performance evaluation on Gaze360 and GFIE datasets demonstrates superior within-dataset performance of the proposed method, reducing angular error (AE) by 0.48° (Gaze360) and 2.95° (GFIE) in static configurations, and 0.59° (Gaze360) and 3.00° (GFIE) in temporal settings compared to prior SOTA methods. Cross-dataset testing shows improvements in AE of more than 1.53° (Gaze360) and 3.99° (GFIE) in both static and temporal settings, validating the robust generalization properties of our approach.
Abstract (translated)
无约束凝视估计是指在不受控制的环境中确定主体视觉注意力方向的过程。凝视估计系统对于多种任务至关重要，例如驾驶员分心监测、考试监考以及现代软件中的辅助功能等。然而，在现实世界场景中，这些系统面临着挑战，部分原因是野外图像分辨率低，另一部分原因则是当前最先进的（SOTA）方法在模拟头眼交互方面不够充分。 本文介绍了一种基于深度学习的方法——DHECA-SuperGaze，该方法通过超分辨率(SR)和双头眼交叉注意(DHECA)模块推进了凝视预测技术。我们的双分支卷积骨干网络处理眼睛图像及多尺度SR头部图像，而提出的DHECA模块则通过跨注意力机制实现提取视觉特征之间的双向细化功能。此外，我们还发现了一个最多样且广泛使用的眼动追踪数据集Gaze360中的关键标注错误，并对这些误标的数据进行了修正。 在Gaze360和GFIE数据集中进行的性能评估表明，与先前的SOTA方法相比，所提出的方法具有优越的内数据集表现，在静态配置下减少了0.48°（Gaze360）和2.95°（GFIE）的角度误差(AE)，在时间设置下则分别减少了0.59°（Gaze360）和3.00°（GFIE）。跨数据集测试显示，在静态和时间设置中，角度误差分别提高了超过1.53°（Gaze360）和3.99°（GFIE），验证了我们方法的稳健泛化特性。
URL
https://arxiv.org/abs/2505.08426
