Abstract
Temporal Action Localization (TAL) has garnered significant attention in information retrieval. Existing supervised or weakly supervised methods heavily rely on labeled temporal boundaries and action categories, which are labor-intensive and time-consuming. Consequently, unsupervised temporal action localization (UTAL) has gained popularity. However, current methods face two main challenges: 1) Classification pre-trained features overly focus on highly discriminative regions; 2) Solely relying on visual modality information makes it difficult to determine contextual boundaries. To address these issues, we propose a CLIP-assisted cross-view audiovisual enhanced UTAL method. Specifically, we introduce visual language pre-training (VLP) and classification pre-training-based collaborative enhancement to avoid excessive focus on highly discriminative regions; we also incorporate audio perception to provide richer contextual boundary information. Finally, we introduce a self-supervised cross-view learning paradigm to achieve multi-view perceptual enhancement without additional annotations. Extensive experiments on two public datasets demonstrate our model's superiority over several state-of-the-art competitors.
Abstract (translated)
时间动作定位（TAL）在信息检索领域引起了广泛的关注。现有的监督或弱监督方法高度依赖于标记的时间边界和动作类别，这需要大量的劳动时间和资源。因此，无监督时间动作定位（UTAL）变得越来越受欢迎。然而，当前的方法面临着两大挑战：1) 预训练的分类特征过于关注高区分性的区域；2) 单纯依靠视觉模态的信息难以确定上下文边界。为了解决这些问题，我们提出了一种CLIP辅助的跨视角音频-视觉增强UTAL方法。具体来说，我们引入了视觉语言预训练（VLP）和基于分类预训练的合作增强机制来避免对高区分性区域的过度关注；同时，我们还融入了音频感知以提供更丰富的上下文边界信息。最后，我们提出了一种自监督跨视角学习范式，在不增加额外标注的情况下实现多视角感知增强。在两个公开数据集上的广泛实验表明，我们的模型优于多个最先进的竞争对手。
URL
https://arxiv.org/abs/2505.23524
