Abstract
Sports analytics has received significant attention from both academia and industry in recent years. Despite the growing interest and efforts in this field, several issues remain unresolved, including (1) data unavailability, (2) lack of an effective trajectory-based framework, and (3) requirement for sufficient supervision labels. In this paper, we present TrajSV, a trajectory-based framework that addresses various issues in existing studies. TrajSV comprises three components: data preprocessing, Clip Representation Network (CRNet), and Video Representation Network (VRNet). The data preprocessing module extracts player and ball trajectories from sports broadcast videos. CRNet utilizes a trajectory-enhanced Transformer module to learn clip representations based on these trajectories. Additionally, VRNet learns video representations by aggregating clip representations and visual features with an encoder-decoder architecture. Finally, a triple contrastive loss is introduced to optimize both video and clip representations in an unsupervised manner. The experiments are conducted on three broadcast video datasets to verify the effectiveness of TrajSV for three types of sports (i.e., soccer, basketball, and volleyball) with three downstream applications (i.e., sports video retrieval, action spotting, and video captioning). The results demonstrate that TrajSV achieves state-of-the-art performance in sports video retrieval, showcasing a nearly 70% improvement. It outperforms baselines in action spotting, achieving state-of-the-art results in 9 out of 17 action categories, and demonstrates a nearly 20% improvement in video captioning. Additionally, we introduce a deployed system along with the three applications based on TrajSV.
Abstract (translated)
近年来，体育数据分析领域受到了学术界和工业界的广泛关注。尽管该领域的兴趣与努力日益增加，但仍存在一些未解决的问题，包括（1）数据不可用性，（2）缺乏有效的基于轨迹的框架，以及（3）需要足够的监督标签。在本文中，我们提出了TrajSV，这是一种基于轨迹的框架，旨在解决现有研究中存在的各种问题。TrajSV包含三个组成部分：数据预处理模块、片段表示网络（CRNet）和视频表示网络（VRNet）。数据预处理模块从体育广播视频中提取球员和球的运动轨迹。CRNet利用增强型Transformer模块根据这些轨迹学习片段表示。此外，VRNet通过编码器-解码器架构聚合片段表示和视觉特征来学习视频表示。最后，引入了三重对比损失以在无监督的方式下优化视频和片段表示。我们在三个广播视频数据集上进行了实验，以验证TrajSV对于三种体育项目（即足球、篮球和排球）的三项下游应用（即体育视频检索、动作识别和视频字幕生成）的有效性。结果表明，TrajSV在体育视频检索方面取得了最先进的性能，并且比基线方法提高了近70%；它在9个类别中的17个类别的动作识别中获得了最佳效果；并且在视频字幕生成上几乎提高了20%的性能。此外，我们还介绍了一个基于TrajSV的部署系统以及上述三个应用。
URL
https://arxiv.org/abs/2508.11569
