Abstract
With the explosive growth of video data in various complex scenarios, quickly retrieving group activities has become an urgent problem. However, many tasks can only retrieve videos focusing on an entire video, not the activity granularity. To solve this problem, we propose a new STVH (spatiotemporal interleaved video hashing) technique for the first time. Through a unified framework, the STVH simultaneously models individual object dynamics and group interactions, capturing the spatiotemporal evolution on both group visual features and positional features. Moreover, in real-life video retrieval scenarios, it may sometimes require activity features, while at other times, it may require visual features of objects. We then further propose a novel M-STVH (multi-focused spatiotemporal video hashing) as an enhanced version to handle this difficult task. The advanced method incorporates hierarchical feature integration through multi-focused representation learning, allowing the model to jointly focus on activity semantics features and object visual features. We conducted comparative experiments on publicly available datasets, and both STVH and M-STVH can achieve excellent results.
Abstract (translated)
随着各种复杂场景中视频数据的爆炸性增长，快速检索群体活动已成为一个紧迫的问题。然而，许多任务只能针对整个视频进行检索，而不能达到活动粒度级别。为了解决这个问题，我们首次提出了一种新的时空交错视频哈希（STVH）技术。通过统一框架，STVH同时建模了个体对象的动态变化和群体互动，捕捉到了群体视觉特征以及位置特征上的时空演变。 然而，在实际的视频检索场景中，有时需要活动特征，而其他时候则可能需要物体的视觉特征。为此，我们进一步提出了一种新颖的多焦点时空视频哈希（M-STVH）作为增强版本来处理这一难题。该高级方法通过多层次表示学习整合了分层特性集成，使模型能够同时关注活动语义特性和对象视觉特性。 我们在公开数据集上进行了比较实验，结果显示STVH和M-STVH都能取得卓越的结果。
URL
https://arxiv.org/abs/2509.00490
