Abstract
Mobile gaze tracking faces a fundamental challenge: maintaining accuracy as users naturally change their postures and device orientations. Traditional calibration approaches, like one-off, fail to adapt to these dynamic conditions, leading to degraded performance over time. We present MAC-Gaze, a Motion-Aware continual Calibration approach that leverages smartphone Inertial measurement unit (IMU) sensors and continual learning techniques to automatically detect changes in user motion states and update the gaze tracking model accordingly. Our system integrates a pre-trained visual gaze estimator and an IMU-based activity recognition model with a clustering-based hybrid decision-making mechanism that triggers recalibration when motion patterns deviate significantly from previously encountered states. To enable accumulative learning of new motion conditions while mitigating catastrophic forgetting, we employ replay-based continual learning, allowing the model to maintain performance across previously encountered motion conditions. We evaluate our system through extensive experiments on the publicly available RGBDGaze dataset and our own 10-hour multimodal MotionGaze dataset (481K+ images, 800K+ IMU readings), encompassing a wide range of postures under various motion conditions including sitting, standing, lying, and walking. Results demonstrate that our method reduces gaze estimation error by 19.9% on RGBDGaze (from 1.73 cm to 1.41 cm) and by 31.7% on MotionGaze (from 2.81 cm to 1.92 cm) compared to traditional calibration approaches. Our framework provides a robust solution for maintaining gaze estimation accuracy in mobile scenarios.
Abstract (translated)
移动眼球追踪面临一个基本挑战：在用户自然改变姿态和设备方向的情况下保持准确性。传统的校准方法，例如一次性校准，在应对这些动态条件时无法适应，导致性能随时间逐渐下降。我们提出了一种称为MAC-Gaze的方法，这是一种基于运动感知的连续校准方法，它利用智能手机惯性测量单元（IMU）传感器和连续学习技术来自动检测用户动作状态的变化，并相应地更新眼球追踪模型。 我们的系统集成了一个预训练的视觉眼球估计器和一个基于IMU的动作识别模型，以及一种基于聚类的混合决策机制，在运动模式显著偏离之前遇到的状态时触发重新校准。为了在适应新运动条件的同时减轻灾难性遗忘的影响，我们采用了基于重放的连续学习方法，使模型能够维持在先前遇到的不同运动条件下的一致性能。 我们在公共可用的RGBDGaze数据集和我们自己的10小时多模态MotionGaze数据集（超过48.1万张图像和80多万次IMU读数）上对系统进行了广泛的实验评估。这些数据集涵盖了坐姿、站姿、躺卧和行走等广泛姿态下的各种运动条件。 结果显示，与传统校准方法相比，我们的方法在RGBDGaze数据集中将眼球估计误差降低了19.9%（从1.73厘米降至1.41厘米），在MotionGaze数据集中则降低了31.7%（从2.81厘米降至1.92厘米）。我们的框架为保持移动场景中的眼球追踪准确性提供了一个稳健的解决方案。
URL
https://arxiv.org/abs/2505.22769
