Abstract
We propose a novel 3D gaze redirection framework that leverages an explicit 3D eyeball structure. Existing gaze redirection methods are typically based on neural radiance fields, which employ implicit neural representations via volume rendering. Unlike these NeRF-based approaches, where the rotation and translation of 3D representations are not explicitly modeled, we introduce a dedicated 3D eyeball structure to represent the eyeballs with 3D Gaussian Splatting (3DGS). Our method generates photorealistic images that faithfully reproduce the desired gaze direction by explicitly rotating and translating the 3D eyeball structure. In addition, we propose an adaptive deformation module that enables the replication of subtle muscle movements around the eyes. Through experiments conducted on the ETH-XGaze dataset, we demonstrate that our framework is capable of generating diverse novel gaze images, achieving superior image quality and gaze estimation accuracy compared to previous state-of-the-art methods.
Abstract (translated)
我们提出了一种新颖的3D目光重定向框架，该框架利用了显式的3D眼球结构。现有的目光重定向方法通常基于神经辐射场（NeRF），这些方法通过体积渲染使用隐式神经表示。与这些基于NeRF的方法不同，其中3D表示的旋转和平移没有被明确建模，我们引入了一个专门的3D眼球结构，并利用三维高斯点阵（3DGS）来表示眼球。我们的方法通过显式地旋转和移动3D眼球结构生成逼真的图像，从而忠实再现所需的目光方向。此外，我们还提出了一种自适应变形模块，使其能够复制眼睛周围细微的肌肉运动。 通过对ETH-XGaze数据集进行实验，我们证明了我们的框架能够在生成多样化的新目光图像方面表现出色，并且在图像质量和目光估计精度上优于现有的最先进的方法。
URL
https://arxiv.org/abs/2508.06136
