Abstract
Gaze estimation, which predicts gaze direction, commonly faces the challenge of interference from complex gaze-irrelevant information in face images. In this work, we propose DMAGaze, a novel gaze estimation framework that exploits information from facial images in three aspects: gaze-relevant global features (disentangled from facial image), local eye features (extracted from cropped eye patch), and head pose estimation features, to improve overall performance. Firstly, we design a new continuous mask-based Disentangler to accurately disentangle gaze-relevant and gaze-irrelevant information in facial images by achieving the dual-branch disentanglement goal through separately reconstructing the eye and non-eye regions. Furthermore, we introduce a new cascaded attention module named Multi-Scale Global Local Attention Module (MS-GLAM). Through a customized cascaded attention structure, it effectively focuses on global and local information at multiple scales, further enhancing the information from the Disentangler. Finally, the global gaze-relevant features disentangled by the upper face branch, combined with head pose and local eye features, are passed through the detection head for high-precision gaze estimation. Our proposed DMAGaze has been extensively validated on two mainstream public datasets, achieving state-of-the-art performance.
Abstract (translated)
眼神估计，即预测凝视方向的任务，通常会面临来自面部图像中与凝视无关的复杂信息干扰。为此，在这项工作中，我们提出了一种新的目光估计框架——DMAGaze，该框架从三个方面利用了面部图像中的信息：相关的全局特征（从面部图像中分离出来）、局部眼睛特征（从裁剪出的眼睛区域提取）和头部姿态估计特征，以提升整体性能。 首先，我们设计了一个新的基于连续掩码的解耦器(Disentangler)，通过分别重建眼部和非眼部区域来准确地将与凝视相关的信息与无关信息分离出来，从而实现双分支解耦的目标。此外，我们还引入了一种新的级联注意力模块——多尺度全局局部注意模块(Multi-Scale Global Local Attention Module, MS-GLAM)。通过定制的级联注意力结构，该模块能够有效关注多个尺度上的全局和局部信息，进一步增强了解耦器提供的信息。最后，上部面部分支解耦出的整体凝视相关特征与头部姿态和局部眼睛特征相结合，并通过检测头进行高精度的目光估计。 我们提出的DMAGaze框架已在两个主流公开数据集上进行了广泛验证，并取得了最先进的性能水平。
URL
https://arxiv.org/abs/2504.11160
