Abstract
Despite decades of research on data collection and model architectures, current gaze estimation models face significant challenges in generalizing across diverse data domains. While recent advances in self-supervised pre-training have shown remarkable potential for improving model generalization in various vision tasks, their effectiveness in gaze estimation remains unexplored due to the geometric nature of the gaze regression task. We propose UniGaze, which leverages large-scale, in-the-wild facial datasets through self-supervised pre-training for gaze estimation. We carefully curate multiple facial datasets that capture diverse variations in identity, lighting, background, and head poses. By directly applying Masked Autoencoder (MAE) pre-training on normalized face images with a Vision Transformer (ViT) backbone, our UniGaze learns appropriate feature representations within the specific input space required by downstream gaze estimation models. Through comprehensive experiments using challenging cross-dataset evaluation and novel protocols, including leave-one-dataset-out and joint-dataset settings, we demonstrate that UniGaze significantly improves generalization across multiple data domains while minimizing reliance on costly labeled data. The source code and pre-trained models will be released upon acceptance.
Abstract (translated)
尽管在数据收集和模型架构方面进行了数十年的研究，当前的视线估计模型仍然面临着跨不同数据域进行泛化的重大挑战。虽然最近在自我监督预训练方面的进展已经在各种视觉任务中展示了显著改善模型泛化的能力，但由于注视回归任务的几何特性，其在注视估计中的有效性尚未被探索。我们提出了UniGaze，该系统利用大规模、现实环境中的人脸数据集通过自我监督预训练来进行注视估计。我们精心策划了多个捕捉身份、光照、背景和头部姿态多种变化的人脸数据集。通过直接将遮罩自动编码器（MAE）预训练应用于具有视觉变换器（ViT）骨干的标准化面部图像，我们的UniGaze学习到了下游注视估计模型所需特定输入空间内的适当特征表示。通过使用具有挑战性的跨数据集评估和包括留一数据集外和联合数据集设置在内的新型协议进行详尽实验，我们证明了UniGaze在多个数据域中的泛化性能得到了显著提升，并且最大限度地减少了对昂贵标注数据的依赖。源代码和预训练模型将在接受后发布。
URL
https://arxiv.org/abs/2502.02307
