Abstract
This paper proposes human-in-the-loop adaptation for Group Activity Feature Learning (GAFL) without group activity annotations. This human-in-the-loop adaptation is employed in a group-activity video retrieval framework to improve its retrieval performance. Our method initially pre-trains the GAF space based on the similarity of group activities in a self-supervised manner, unlike prior work that classifies videos into pre-defined group activity classes in a supervised learning manner. Our interactive fine-tuning process updates the GAF space to allow a user to better retrieve videos similar to query videos given by the user. In this fine-tuning, our proposed data-efficient video selection process provides several videos, which are selected from a video database, to the user in order to manually label these videos as positive or negative. These labeled videos are used to update (i.e., fine-tune) the GAF space, so that the positive and negative videos move closer to and farther away from the query videos through contrastive learning. Our comprehensive experimental results on two team sports datasets validate that our method significantly improves the retrieval performance. Ablation studies also demonstrate that several components in our human-in-the-loop adaptation contribute to the improvement of the retrieval performance. Code: this https URL.
Abstract (translated)
本文提出了一种无需群体活动标注的人机交互适应方法，用于群体活动特征学习（GAFL）。该人机交互适应技术被应用到一个群体活动视频检索框架中，以提升其检索性能。我们的方法首先通过自监督方式基于群体活动的相似性来预训练GAF空间，与以往的研究不同的是，后者在有监督的学习模式下将视频分类为预先定义好的群体活动类别。 我们提出的方法包括一种交互式的微调过程，该过程更新GAF空间以使用户能够更好地检索到与其查询视频类似的视频。在此微调过程中，我们的高效数据选择流程从一个视频数据库中选取几个视频，并提供给用户手动标注这些视频是正样本还是负样本。之后，根据这些标签更新（即细化）GAF空间，从而通过对比学习方法让正样本和负样本与查询视频的距离更近或更远。 我们在两个团队运动数据集上的全面实验结果验证了我们的方法显著提升了检索性能。消融研究表明我们的人机交互适应中的几个组件对提高检索性能有贡献作用。代码链接：[请将URL替换为实际提供的链接]。
URL
https://arxiv.org/abs/2602.03157
