Abstract
Gaze Target Detection (GTD), i.e., determining where a person is looking within a scene from an external viewpoint, is a challenging task, particularly in 3D space. Existing approaches heavily rely on analyzing the person's appearance, primarily focusing on their face to predict the gaze target. This paper presents a novel approach to tackle this problem by utilizing the person's upper-body pose and available depth maps to extract a 3D gaze direction and employing a multi-stage or an end-to-end pipeline to predict the gazed target. When predicted accurately, the human body pose can provide valuable information about the head pose, which is a good approximation of the gaze direction, as well as the position of the arms and hands, which are linked to the activity the person is performing and the objects they are likely focusing on. Consequently, in addition to performing gaze estimation in 3D, we are also able to perform GTD simultaneously. We demonstrate state-of-the-art results on the most comprehensive publicly accessible 3D gaze target detection dataset without requiring images of the person's face, thus promoting privacy preservation in various application contexts. The code is available at this https URL.
Abstract (translated)
凝视目标检测（GTD）是一种具有挑战性的任务，特别是在三维空间中。现有的方法很大程度上依赖于分析人的外貌，主要集中在其脸上预测 gaze 目标。本文提出了一种新方法来解决这个问题，通过利用人的上半身姿态和可用的深度图来提取 3D gaze 方向，并采用多阶段或端到端管道来预测 gaze 目标。当预测准确时，人体姿态可以提供关于头姿的信息，这是 gaze 方向的近似，以及手臂和手的位置，这些与人们正在进行的活动和他们正在关注的物体有关。因此，我们在进行 3D 凝视估计的同时，也能够进行 GTD。我们在没有要求预测人面图像的公开可访问的三维凝视目标检测数据集上证明了最先进的结果，从而在各种应用场景中促进了隐私保护。代码可在此处访问：https://url.cn/xyz6h
URL
https://arxiv.org/abs/2409.17886
