Abstract
Human-machine interaction through augmented reality (AR) and virtual reality (VR) is increasingly prevalent, requiring accurate and efficient gaze estimation which hinges on the accuracy of eye segmentation to enable smooth user experiences. We introduce EyeSeg, a novel eye segmentation framework designed to overcome key challenges that existing approaches struggle with: motion blur, eyelid occlusion, and train-test domain gaps. In these situations, existing models struggle to extract robust features, leading to suboptimal performance. Noting that these challenges can be generally quantified by uncertainty, we design EyeSeg as an uncertainty-aware eye segmentation framework for AR/VR wherein we explicitly model the uncertainties by performing Bayesian uncertainty learning of a posterior under the closed set prior. Theoretically, we prove that a statistic of the learned posterior indicates segmentation uncertainty levels and empirically outperforms existing methods in downstream tasks, such as gaze estimation. EyeSeg outputs an uncertainty score and the segmentation result, weighting and fusing multiple gaze estimates for robustness, which proves to be effective especially under motion blur, eyelid occlusion and cross-domain challenges. Moreover, empirical results suggest that EyeSeg achieves segmentation improvements of MIoU, E1, F1, and ACC surpassing previous approaches. The code is publicly available at this https URL.
Abstract (translated)
通过增强现实（AR）和虚拟现实（VR）的人机交互越来越普遍，这要求进行准确且高效的视线估计，而这又依赖于眼睛分割的准确性以实现流畅的用户体验。我们引入了EyeSeg，这是一个新型的眼睛分割框架，旨在克服现有方法在运动模糊、眼睑遮挡以及训练与测试领域差异等方面的挑战。在这种情况下，现有的模型难以提取出稳健的功能特征，导致性能不佳。 注意到这些挑战可以一般性地量化为不确定性，我们将EyeSeg设计成一个针对AR/VR的不确定性感知的眼睛分割框架，在其中我们通过执行闭集先验下的贝叶斯不确定性学习来显式建模不确定性。理论上，我们证明了所学后验统计量指示了分割不确定性的水平，并在下游任务（如视线估计）中实证性地优于现有方法。 EyeSeg输出一个不确定性评分和分割结果，在多个眼睛位置的估计中进行加权融合以增强鲁棒性，这尤其在运动模糊、眼睑遮挡及跨领域挑战下效果显著。此外，实验结果显示EyeSeg在MIoU（平均交并比）、E1、F1和ACC（准确率）等指标上的分割改进超过了以往的方法。 EyeSeg的代码已公开，可在此网址访问：[请将此链接替换为实际提供的网址]。
URL
https://arxiv.org/abs/2507.09649
