Gaze_Estimation
-
Toward Super-Resolution for Appearance-Based Gaze Estimation
2023-03-17 17:40:32Galen O'Shea, Majid KomeiliAbstract
Gaze tracking is a valuable tool with a broad range of applications in various fields, including medicine, psychology, virtual reality, marketing, and safety. Therefore, it is essential to have gaze tracking software that is cost-efficient and high-performing. Accurately predicting gaze remains a difficult task, particularly in real-world situations where images are affected by motion blur, video compression, and noise. Super-resolution has been shown to improve image quality from a visual perspective. This work examines the usefulness of super-resolution for improving appearance-based gaze tracking. We show that not all SR models preserve the gaze direction. We propose a two-step framework based on SwinIR super-resolution model. The proposed method consistently outperforms the state-of-the-art, particularly in scenarios involving low-resolution or degraded images. Furthermore, we examine the use of super-resolution through the lens of self-supervised learning for gaze prediction. Self-supervised learning aims to learn from unlabelled data to reduce the amount of required labeled data for downstream tasks. We propose a novel architecture called SuperVision by fusing an SR backbone network to a ResNet18 (with some skip connections). The proposed SuperVision method uses 5x less labeled data and yet outperforms, by 15%, the state-of-the-art method of GazeTR which uses 100% of training data.
Abstract (translated)
注视 tracking是一类重要的工具,在各种领域都有广泛的应用,包括医学、心理学、虚拟现实、营销和安全等。因此,拥有高效且性能优秀的注视 tracking软件是至关重要的。精确预测注视仍然是一项具有挑战性的任务,特别是在图像受到运动模糊、视频压缩和噪声影响的实际场景下。超分辨率已经被证明可以改善图像质量。本研究探讨了超分辨率如何用于改善外观基于注视 tracking。我们表明,不是所有的SR模型都保留注视方向。我们提出了基于 SwinIR 超分辨率模型的两步框架。该框架 consistently outperforms the state-of-the-art,特别是在涉及低分辨率或图像恶化的场景下。此外,我们考虑了通过自监督学习视角使用超分辨率进行注视预测的必要性。自监督学习旨在从未标记数据学习以减少后续任务所需的标记数据量。我们提出了一种名为SuperVision的新架构,通过将SR基线网络与ResNet18融合(并添加一些跳过连接)来实现。该SuperVision方法使用更少的标记数据,但仍比使用训练数据的100%的 gazeTR方法表现更好,提高了15%。
URL
https://arxiv.org/abs/2303.10151
PDF
-
Confidence-aware 3D Gaze Estimation and Evaluation Metric
2023-03-17 15:44:44Qiaojie Zheng, Jiucai Zhang, Amy Zhang, Xiaoli ZhangAbstract
Deep learning appearance-based 3D gaze estimation is gaining popularity due to its minimal hardware requirements and being free of constraint. Unreliable and overconfident inferences, however, still limit the adoption of this gaze estimation method. To address the unreliable and overconfident issues, we introduce a confidence-aware model that predicts uncertainties together with gaze angle estimations. We also introduce a novel effectiveness evaluation method based on the causality between eye feature degradation and the rise in inference uncertainty to assess the uncertainty estimation. Our confidence-aware model demonstrates reliable uncertainty estimations while providing angular estimation accuracies on par with the state-of-the-art. Compared with the existing statistical uncertainty-angular-error evaluation metric, the proposed effectiveness evaluation approach can more effectively judge inferred uncertainties' performance at each prediction.
Abstract (translated)
Deep learning 基于外观的三维目光估计因其硬件要求最小且不受限制而越来越受欢迎。然而,不可靠的和过于自信的决策仍然限制着这种方法的采用。为了解决这些不可靠和过于自信的问题,我们引入了一种具有自我意识的模型,它可以同时预测不确定性和目光角度估计。我们还介绍了一种基于 eye feature 退化和推断不确定性的上升的因果关系的新有效性评估方法,以评估不确定性估计。我们的具有自我意识的模型表现出可靠的不确定性估计,同时提供与当前最先进的准确性相当的角估计精度。与现有的统计不确定性-角误差评估度量相比,我们提出的有效性评估方法可以更有效地评估推断不确定性在每个预测中的性能。
URL
https://arxiv.org/abs/2303.10062
PDF
-
Towards Precision in Appearance-based Gaze Estimation in the Wild
2023-02-05 10:09:35Murthy L.R.D., Abhishek Mukhopadhyay, Shambhavi Aggarwal, Ketan Anand, Pradipta BiswasAbstract
Appearance-based gaze estimation systems have shown great progress recently, yet the performance of these techniques depend on the datasets used for training. Most of the existing gaze estimation datasets setup in interactive settings were recorded in laboratory conditions and those recorded in the wild conditions display limited head pose and illumination variations. Further, we observed little attention so far towards precision evaluations of existing gaze estimation approaches. In this work, we present a large gaze estimation dataset, PARKS-Gaze, with wider head pose and illumination variation and with multiple samples for a single Point of Gaze (PoG). The dataset contains 974 minutes of data from 28 participants with a head pose range of 60 degrees in both yaw and pitch directions. Our within-dataset and cross-dataset evaluations and precision evaluations indicate that the proposed dataset is more challenging and enable models to generalize on unseen participants better than the existing in-the-wild datasets. The project page can be accessed here: this https URL
Abstract (translated)
appearance-based gaze estimation systems 在最近取得了巨大的进展,但这些技巧的性能取决于用于训练的 datasets。大多数现有的 gaze estimation dataset 在互动设置中设置是在实验室条件下录制的,而在野生条件下录制的dataset 显示 head pose 和照明变化非常有限。此外,我们观察到迄今为止 little 关注于 precision 评估 existing gaze estimation approaches。在本研究中,我们提出了一个大型 gaze estimation dataset,PARKS-Gaze,具有更广泛的 head pose 和照明变化,并且具有多个样本,以单点 gaze (PoG)。dataset 包含 974 分钟的数据,来自 28 名参与者,其 head pose 范围在 yaw 和 pitch 方向上为 60 度。我们的dataset 内部和跨dataset 评估以及 precision 评估表明, proposed dataset 更具挑战性,使模型能够对未观测到的参与者更普遍地泛化。项目页面可访问此处:这个 https URL。
URL
https://arxiv.org/abs/2302.02353
PDF
-
Accurate Gaze Estimation using an Active-gaze Morphable Model
Abstract
Rather than regressing gaze direction directly from images, we show that adding a 3D shape model can: i) improve gaze estimation accuracy, ii) perform well with lower resolution inputs and iii) provide a richer understanding of the eye-region and its constituent gaze system. Specifically, we use an `eyes and nose' 3D morphable model (3DMM) to capture the eye-region 3D facial geometry and appearance and we equip this with a geometric vergence model of gaze to give an `active-gaze 3DMM'. We show that our approach achieves state-of-the-art results on the Eyediap dataset and we present an ablation study. Our method can learn with only the ground truth gaze target point and the camera parameters, without access to the ground truth gaze origin points, thus widening the applicability of our approach compared to other methods.
Abstract (translated)
URL
https://arxiv.org/abs/2301.13186
PDF
-
It's Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation
2023-01-09 18:39:09Xucong Zhang, Yusuke Sugano, Mario Fritz, Andreas BullingAbstract
Eye gaze is an important non-verbal cue for human affect analysis. Recent gaze estimation work indicated that information from the full face region can benefit performance. Pushing this idea further, we propose an appearance-based method that, in contrast to a long-standing line of work in computer vision, only takes the full face image as input. Our method encodes the face image using a convolutional neural network with spatial weights applied on the feature maps to flexibly suppress or enhance information in different facial regions. Through extensive evaluation, we show that our full-face method significantly outperforms the state of the art for both 2D and 3D gaze estimation, achieving improvements of up to 14.3% on MPIIGaze and 27.7% on EYEDIAP for person-independent 3D gaze estimation. We further show that this improvement is consistent across different illumination conditions and gaze directions and particularly pronounced for the most challenging extreme head poses.
Abstract (translated)
URL
https://arxiv.org/abs/1611.08860
PDF
-
NeRF-Gaze: A Head-Eye Redirection Parametric Model for Gaze Estimation
2022-12-30 13:52:28Pengwei Yin, Jiawu Dai, Jingjing Wang, Di Xie, Shiliang PuAbstract
Gaze estimation is the fundamental basis for many visual tasks. Yet, the high cost of acquiring gaze datasets with 3D annotations hinders the optimization and application of gaze estimation models. In this work, we propose a novel Head-Eye redirection parametric model based on Neural Radiance Field, which allows dense gaze data generation with view consistency and accurate gaze direction. Moreover, our head-eye redirection parametric model can decouple the face and eyes for separate neural rendering, so it can achieve the purpose of separately controlling the attributes of the face, identity, illumination, and eye gaze direction. Thus diverse 3D-aware gaze datasets could be obtained by manipulating the latent code belonging to different face attributions in an unsupervised manner. Extensive experiments on several benchmarks demonstrate the effectiveness of our method in domain generalization and domain adaptation for gaze estimation tasks.
Abstract (translated)
URL
https://arxiv.org/abs/2212.14710
PDF
-
Weakly-Supervised Gaze Estimation from Synthetic Views
2022-12-06 14:15:17Evangelos Ververas, Polydefkis Gkagkos, Jiankang Deng, Jia Guo, Michail Christos Doukas, Stefanos ZafeiriouAbstract
3D gaze estimation is most often tackled as learning a direct mapping between input images and the gaze vector or its spherical coordinates. Recently, it has been shown that pose estimation of the face, body and hands benefits from revising the learning target from few pose parameters to dense 3D coordinates. In this work, we leverage this observation and propose to tackle 3D gaze estimation as regression of 3D eye meshes. We overcome the absence of compatible ground truth by fitting a rigid 3D eyeball template on existing gaze datasets and propose to improve generalization by making use of widely available in-the-wild face images. To this end, we propose an automatic pipeline to retrieve robust gaze pseudo-labels from arbitrary face images and design a multi-view supervision framework to balance their effect during training. In our experiments, our method achieves improvement of 30% compared to state-of-the-art in cross-dataset gaze estimation, when no ground truth data are available for training, and 7% when they are. We make our project publicly available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2212.02997
PDF
-
One Eye is All You Need: Lightweight Ensembles for Gaze Estimation with Single Encoders
2022-11-22 01:12:31Rishi Athavale, Lakshmi Sritan Motati, Rohan KalahastyAbstract
Gaze estimation has grown rapidly in accuracy in recent years. However, these models often fail to take advantage of different computer vision (CV) algorithms and techniques (such as small ResNet and Inception networks and ensemble models) that have been shown to improve results for other CV problems. Additionally, most current gaze estimation models require the use of either both eyes or an entire face, whereas real-world data may not always have both eyes in high resolution. Thus, we propose a gaze estimation model that implements the ResNet and Inception model architectures and makes predictions using only one eye image. Furthermore, we propose an ensemble calibration network that uses the predictions from several individual architectures for subject-specific predictions. With the use of lightweight architectures, we achieve high performance on the GazeCapture dataset with very low model parameter counts. When using two eyes as input, we achieve a prediction error of 1.591 cm on the test set without calibration and 1.439 cm with an ensemble calibration model. With just one eye as input, we still achieve an average prediction error of 2.312 cm on the test set without calibration and 1.951 cm with an ensemble calibration model. We also notice significantly lower errors on the right eye images in the test set, which could be important in the design of future gaze estimation-based tools.
Abstract (translated)
URL
https://arxiv.org/abs/2211.11936
PDF
-
Predicting Eye Gaze Location on Websites
2022-11-15 11:55:46Ciheng Zhang, Decky Aspandi, Steffen StaabAbstract
World-wide-web, with the website and webpage as the main interface, facilitates the dissemination of important information. Hence it is crucial to optimize them for better user interaction, which is primarily done by analyzing users' behavior, especially users' eye-gaze locations. However, gathering these data is still considered to be labor and time intensive. In this work, we enable the development of automatic eye-gaze estimations given a website screenshots as the input. This is done by the curation of a unified dataset that consists of website screenshots, eye-gaze heatmap and website's layout information in the form of image and text masks. Our pre-processed dataset allows us to propose an effective deep learning-based model that leverages both image and text spatial location, which is combined through attention mechanism for effective eye-gaze prediction. In our experiment, we show the benefit of careful fine-tuning using our unified dataset to improve the accuracy of eye-gaze predictions. We further observe the capability of our model to focus on the targeted areas (images and text) to achieve high accuracy. Finally, the comparison with other alternatives shows the state-of-the-art result of our model establishing the benchmark for the eye-gaze prediction task.
Abstract (translated)
URL
https://arxiv.org/abs/2211.08074
PDF
-
Contrastive Weighted Learning for Near-Infrared Gaze Estimation
2022-11-06 10:03:23Adam LeeAbstract
Appearance-based gaze estimation has been very successful with the use of deep learning. Many following works improved domain generalization for gaze estimation. However, even though there has been much progress in domain generalization for gaze estimation, most of the recent work have been focused on cross-dataset performance -- accounting for different distributions in illuminations, head pose, and lighting. Although improving gaze estimation in different distributions of RGB images is important, near-infrared image based gaze estimation is also critical for gaze estimation in dark settings. Also there are inherent limitations relying solely on supervised learning for regression tasks. This paper contributes to solving these problems and proposes GazeCWL, a novel framework for gaze estimation with near-infrared images using contrastive learning. This leverages adversarial attack techniques for data augmentation and a novel contrastive loss function specifically for regression tasks that effectively clusters the features of different samples in the latent space. Our model outperforms previous domain generalization models in infrared image based gaze estimation and outperforms the baseline by 45.6\% while improving the state-of-the-art by 8.6\%, we demonstrate the efficacy of our method.
Abstract (translated)
URL
https://arxiv.org/abs/2211.03073
PDF
-
An Effective Deep Network for Head Pose Estimation without Keypoints
2022-10-25 01:57:04Chien Thai, Viet Tran, Minh Bui, Huong Ninh, Hai TranAbstract
Human head pose estimation is an essential problem in facial analysis in recent years that has a lot of computer vision applications such as gaze estimation, virtual reality, and driver assistance. Because of the importance of the head pose estimation problem, it is necessary to design a compact model to resolve this task in order to reduce the computational cost when deploying on facial analysis-based applications such as large camera surveillance systems, AI cameras while maintaining accuracy. In this work, we propose a lightweight model that effectively addresses the head pose estimation problem. Our approach has two main steps. 1) We first train many teacher models on the synthesis dataset - 300W-LPA to get the head pose pseudo labels. 2) We design an architecture with the ResNet18 backbone and train our proposed model with the ensemble of these pseudo labels via the knowledge distillation process. To evaluate the effectiveness of our model, we use AFLW-2000 and BIWI - two real-world head pose datasets. Experimental results show that our proposed model significantly improves the accuracy in comparison with the state-of-the-art head pose estimation methods. Furthermore, our model has the real-time speed of $\sim$300 FPS when inferring on Tesla V100.
Abstract (translated)
URL
https://arxiv.org/abs/2210.13705
PDF
-
Contrastive Representation Learning for Gaze Estimation
2022-10-24 17:01:18Swati Jindal, Roberto ManduchiAbstract
Self-supervised learning (SSL) has become prevalent for learning representations in computer vision. Notably, SSL exploits contrastive learning to encourage visual representations to be invariant under various image transformations. The task of gaze estimation, on the other hand, demands not just invariance to various appearances but also equivariance to the geometric transformations. In this work, we propose a simple contrastive representation learning framework for gaze estimation, named Gaze Contrastive Learning (GazeCLR). GazeCLR exploits multi-view data to promote equivariance and relies on selected data augmentation techniques that do not alter gaze directions for invariance learning. Our experiments demonstrate the effectiveness of GazeCLR for several settings of the gaze estimation task. Particularly, our results show that GazeCLR improves the performance of cross-domain gaze estimation and yields as high as 17.2% relative improvement. Moreover, the GazeCLR framework is competitive with state-of-the-art representation learning methods for few-shot evaluation. The code and pre-trained models are available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2210.13404
PDF
-
Jitter Does Matter: Adapting Gaze Estimation to New Domains
2022-10-05 08:20:41Ruicong Liu, Yiwei Bao, Mingjie Xu, Haofei Wang, Yunfei Liu, Feng LuAbstract
Deep neural networks have demonstrated superior performance on appearance-based gaze estimation tasks. However, due to variations in person, illuminations, and background, performance degrades dramatically when applying the model to a new domain. In this paper, we discover an interesting gaze jitter phenomenon in cross-domain gaze estimation, i.e., the gaze predictions of two similar images can be severely deviated in target domain. This is closely related to cross-domain gaze estimation tasks, but surprisingly, it has not been noticed yet previously. Therefore, we innovatively propose to utilize the gaze jitter to analyze and optimize the gaze domain adaptation task. We find that the high-frequency component (HFC) is an important factor that leads to jitter. Based on this discovery, we add high-frequency components to input images using the adversarial attack and employ contrastive learning to encourage the model to obtain similar representations between original and perturbed data, which reduces the impacts of HFC. We evaluate the proposed method on four cross-domain gaze estimation tasks, and experimental results demonstrate that it significantly reduces the gaze jitter and improves the gaze estimation performance in target domains.
Abstract (translated)
URL
https://arxiv.org/abs/2210.02082
PDF
-
LatentGaze: Cross-Domain Gaze Estimation through Gaze-Aware Analytic Latent Code Manipulation
2022-09-21 08:05:53Isack Lee, Jun-Seok Yun, Hee Hyeon Kim, Youngju Na, Seok Bong YooAbstract
Although recent gaze estimation methods lay great emphasis on attentively extracting gaze-relevant features from facial or eye images, how to define features that include gaze-relevant components has been ambiguous. This obscurity makes the model learn not only gaze-relevant features but also irrelevant ones. In particular, it is fatal for the cross-dataset performance. To overcome this challenging issue, we propose a gaze-aware analytic manipulation method, based on a data-driven approach with generative adversarial network inversion's disentanglement characteristics, to selectively utilize gaze-relevant features in a latent code. Furthermore, by utilizing GAN-based encoder-generator process, we shift the input image from the target domain to the source domain image, which a gaze estimator is sufficiently aware. In addition, we propose gaze distortion loss in the encoder that prevents the distortion of gaze information. The experimental results demonstrate that our method achieves state-of-the-art gaze estimation accuracy in a cross-domain gaze estimation tasks. This code is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2209.10171
PDF
-
HAZE-Net: High-Frequency Attentive Super-Resolved Gaze Estimation in Low-Resolution Face Images
2022-09-21 07:57:07Jun-Seok Yun, Youngju Na, Hee Hyeon Kim, Hyung-Il Kim, Seok Bong YooAbstract
Although gaze estimation methods have been developed with deep learning techniques, there has been no such approach as aim to attain accurate performance in low-resolution face images with a pixel width of 50 pixels or less. To solve a limitation under the challenging low-resolution conditions, we propose a high-frequency attentive super-resolved gaze estimation network, i.e., HAZE-Net. Our network improves the resolution of the input image and enhances the eye features and those boundaries via a proposed super-resolution module based on a high-frequency attention block. In addition, our gaze estimation module utilizes high-frequency components of the eye as well as the global appearance map. We also utilize the structural location information of faces to approximate head pose. The experimental results indicate that the proposed method exhibits robust gaze estimation performance even in low-resolution face images with 28x28 pixels. The source code of this work is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2209.10167
PDF
-
FreeGaze: Resource-efficient Gaze Estimation via Frequency Domain Contrastive Learning
2022-09-14 14:51:52Lingyu Du, Guohao LanAbstract
Gaze estimation is of great importance to many scientific fields and daily applications, ranging from fundamental research in cognitive psychology to attention-aware mobile systems. While recent advancements in deep learning have yielded remarkable successes in building highly accurate gaze estimation systems, the associated high computational cost and the reliance on large-scale labeled gaze data for supervised learning place challenges on the practical use of existing solutions. To move beyond these limitations, we present FreeGaze, a resource-efficient framework for unsupervised gaze representation learning. FreeGaze incorporates the frequency domain gaze estimation and the contrastive gaze representation learning in its design. The former significantly alleviates the computational burden in both system calibration and gaze estimation, and dramatically reduces the system latency; while the latter overcomes the data labeling hurdle of existing supervised learning-based counterparts, and ensures efficient gaze representation learning in the absence of gaze label. Our evaluation on two gaze estimation datasets shows that FreeGaze can achieve comparable gaze estimation accuracy with existing supervised learning-based approach, while enabling up to 6.81 and 1.67 times speedup in system calibration and gaze estimation, respectively.
Abstract (translated)
URL
https://arxiv.org/abs/2209.06692
PDF
-
What You See is What You Grasp: User-Friendly Grasping Guided by Near-eye-tracking
2022-09-13 16:14:06Shaochen Wang, Wei Zhang, Zhangli Zhou, Jiaxi Cao, Ziyang Chen, Kang Chen, Bin Li, Zhen KanAbstract
This work presents a next-generation human-robot interface that can infer and realize the user's manipulation intention via sight only. Specifically, we develop a system that integrates near-eye-tracking and robotic manipulation to enable user-specified actions (e.g., grasp, pick-and-place, etc), where visual information is merged with human attention to create a mapping for desired robot actions. To enable sight guided manipulation, a head-mounted near-eye-tracking device is developed to track the eyeball movements in real-time, so that the user's visual attention can be identified. To improve the grasping performance, a transformer based grasp model is then developed. Stacked transformer blocks are used to extract hierarchical features where the volumes of channels are expanded at each stage while squeezing the resolution of feature maps. Experimental validation demonstrates that the eye-tracking system yields low gaze estimation error and the grasping system yields promising results on multiple grasping datasets. This work is a proof of concept for gaze interaction-based assistive robot, which holds great promise to help the elder or upper limb disabilities in their daily lives. A demo video is available at \url{this https URL}.
Abstract (translated)
URL
https://arxiv.org/abs/2209.06122
PDF
-
In the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimation
2022-08-08 23:25:05Bolin Lai, Miao Liu, Fiona Ryan, James RehgAbstract
In this paper, we present the first transformer-based model to address the challenging problem of egocentric gaze estimation. We observe that the connection between the global scene context and local visual information is vital for localizing the gaze fixation from egocentric video frames. To this end, we design the transformer encoder to embed the global context as one additional visual token and further propose a novel Global-Local Correlation (GLC) module to explicitly model the correlation of the global token and each local token. We validate our model on two egocentric video datasets - EGTEA Gaze+ and Ego4D. Our detailed ablation studies demonstrate the benefits of our method. In addition, our approach exceeds previous state-of-the-arts by a large margin. We also provide additional visualizations to support our claim that global-local correlation serves a key representation for predicting gaze fixation from egocentric videos. More details can be found in our website (this https URL).
Abstract (translated)
URL
https://arxiv.org/abs/2208.04464
PDF
-
Gaze Estimation Approach Using Deep Differential Residual Network
2022-08-08 17:37:07Longzhao Huang, Yujie Li, Xu Wang, Haoyu Wang, Ahmed Bouridane, Ahmad ChaddadAbstract
Gaze estimation, which is a method to determine where a person is looking at given the person's full face, is a valuable clue for understanding human intention. Similarly to other domains of computer vision, deep learning (DL) methods have gained recognition in the gaze estimation domain. However, there are still gaze calibration problems in the gaze estimation domain, thus preventing existing methods from further improving the performances. An effective solution is to directly predict the difference information of two human eyes, such as the differential network (Diff-Nn). However, this solution results in a loss of accuracy when using only one inference image. We propose a differential residual model (DRNet) combined with a new loss function to make use of the difference information of two eye images. We treat the difference information as auxiliary information. We assess the proposed model (DRNet) mainly using two public datasets (1) MpiiGaze and (2) Eyediap. Considering only the eye features, DRNet outperforms the state-of-the-art gaze estimation methods with $angular-error$ of 4.57 and 6.14 using MpiiGaze and Eyediap datasets, respectively. Furthermore, the experimental results also demonstrate that DRNet is extremely robust to noise images.
Abstract (translated)
URL
https://arxiv.org/abs/2208.04298
PDF
-
RAZE: Region Guided Self-Supervised Gaze Representation Learning
2022-08-04 06:23:49Neeru Dubey, Shreya Ghosh, Abhinav DhallAbstract
Automatic eye gaze estimation is an important problem in vision based assistive technology with use cases in different emerging topics such as augmented reality, virtual reality and human-computer interaction. Over the past few years, there has been an increasing interest in unsupervised and self-supervised learning paradigms as it overcomes the requirement of large scale annotated data. In this paper, we propose RAZE, a Region guided self-supervised gAZE representation learning framework which leverage from non-annotated facial image data. RAZE learns gaze representation via auxiliary supervision i.e. pseudo-gaze zone classification where the objective is to classify visual field into different gaze zones (i.e. left, right and center) by leveraging the relative position of pupil-centers. Thus, we automatically annotate pseudo gaze zone labels of 154K web-crawled images and learn feature representations via `Ize-Net' framework. `Ize-Net' is a capsule layer based CNN architecture which can efficiently capture rich eye representation. The discriminative behaviour of the feature representation is evaluated on four benchmark datasets: CAVE, TabletGaze, MPII and RT-GENE. Additionally, we evaluate the generalizability of the proposed network on two other downstream task (i.e. driver gaze estimation and visual attention estimation) which demonstrate the effectiveness of the learnt eye gaze representation.
Abstract (translated)
URL
https://arxiv.org/abs/2208.02485
PDF
