Abstract
The Apple Vision Pro is equipped with accurate eye-tracking capabilities, yet the privacy restrictions on the device prevent direct access to continuous user gaze data. This study introduces iTrace, a novel application that overcomes these limitations through click-based gaze extraction techniques, including manual methods like a pinch gesture, and automatic approaches utilizing dwell control or a gaming controller. We developed a system with a client-server architecture that captures the gaze coordinates and transforms them into dynamic heatmaps for video and spatial eye tracking. The system can generate individual and averaged heatmaps, enabling analysis of personal and collective attention patterns. To demonstrate its effectiveness and evaluate the usability and performance, a study was conducted with two groups of 10 participants, each testing different clicking methods. The 8BitDo controller achieved higher average data collection rates at 14.22 clicks/s compared to 0.45 clicks/s with dwell control, enabling significantly denser heatmap visualizations. The resulting heatmaps reveal distinct attention patterns, including concentrated focus in lecture videos and broader scanning during problem-solving tasks. By allowing dynamic attention visualization while maintaining a high gaze precision of 91 %, iTrace demonstrates strong potential for a wide range of applications in educational content engagement, environmental design evaluation, marketing analysis, and clinical cognitive assessment. Despite the current gaze data restrictions on the Apple Vision Pro, we encourage developers to use iTrace only in research settings.
Abstract (translated)
苹果Vision Pro配备了精确的眼动追踪功能，但由于设备的隐私限制，无法直接访问连续的用户注视数据。本研究引入了一种名为iTrace的新应用，该应用通过基于点击的眼动提取技术克服了这些限制，包括手动方法（如捏合手势）和自动方法（利用停留控制或游戏手柄）。我们开发了一个采用客户端-服务器架构的系统，可以捕获注视坐标并将其转换为动态热图，用于视频和空间眼动追踪。该系统能够生成个人和平均化的热图，从而分析个人和群体的关注模式。 为了展示其有效性，并评估其可用性和性能，本研究对两组各10名参与者进行了测试，每组使用不同的点击方法。8BitDo控制器在数据收集率方面表现更佳，平均每秒为14.22次点击（相比之下，停留控制仅为0.45次/秒），这使得生成的热图可视化更为密集。 生成的热图揭示了明显不同的注意力模式：在讲座视频中表现为集中关注，在问题解决任务中则表现为广泛的扫描。通过允许动态注意力可视化，并保持91%的眼动精确度，iTrace显示出在教育内容参与、环境设计评估、市场分析和临床认知评估等广泛领域的强大潜力。 尽管当前的苹果Vision Pro对注视数据有限制，我们鼓励开发者仅在研究环境中使用iTrace。
URL
https://arxiv.org/abs/2508.12268
