Abstract
Sports analysis and viewing play a pivotal role in the current sports domain, offering significant value not only to coaches and athletes but also to fans and the media. In recent years, the rapid development of virtual reality (VR) and augmented reality (AR) technologies have introduced a new platform for watching games. Visualization of sports competitions in VR/AR represents a revolutionary technology, providing audiences with a novel immersive viewing experience. However, there is still a lack of related research in this area. In this work, we present for the first time a comprehensive system for sports competition analysis and real-time visualization on VR/AR platforms. First, we utilize multiview LiDARs and cameras to collect multimodal game data. Subsequently, we propose a framework for multi-player tracking and pose estimation based on a limited amount of supervised data, which extracts precise player positions and movements from point clouds and images. Moreover, we perform avatar modeling of players to obtain their 3D models. Ultimately, using these 3D player data, we conduct competition analysis and real-time visualization on VR/AR. Extensive quantitative experiments demonstrate the accuracy and robustness of our multi-player tracking and pose estimation framework. The visualization results showcase the immense potential of our sports visualization system on the domain of watching games on VR/AR devices. The multimodal competition dataset we collected and all related code will be released soon.
Abstract (translated)
体育分析和实时观看在当前体育领域中扮演着关键角色，为教练、运动员和球迷以及媒体提供了宝贵的价值。近年来，虚拟现实（VR）和增强现实（AR）技术的快速发展为观看比赛提供了新的平台。在VR/AR中可视化体育比赛代表了一种革命性的技术，为观众提供了新颖的沉浸式观看体验。然而，在这个领域仍然缺乏相关研究。在这项工作中，我们首次提出了一个完整的体育比赛分析及实时可视化在VR/AR平台上的系统。首先，我们利用多视角 LiDAR 和相机收集多模态游戏数据。接着，我们提出了一种基于有限监督数据的多玩家跟踪和姿态估计框架，从点云和图像中提取精确的球员位置和运动。此外，我们还为玩家创建了3D模型。最后，利用这些3D玩家数据，我们在VR/AR上进行比赛分析和实时可视化。大量的定量实验证明了我们多玩家跟踪和姿态估计框架的准确性和稳健性。可视化结果展示了我们在VR/AR设备领域观看游戏的巨大潜力。我们收集的多模态比赛数据和所有相关的代码即将发布。
URL
https://arxiv.org/abs/2405.01112
