Abstract
Detecting temporal extents of human actions in videos is a challenging computer vision problem that require detailed manual supervision including frame-level labels. This expensive annotation process limits deploying action detectors on a limited number of categories. We propose a novel action recognition method, called WSGN, that can learn to detect actions from "weak supervision", video-level labels. WSGN learns to exploit both video-specific and dataset-wide statistics to predict relevance of each frame to an action category. We show that a combination of the local and global channels leads to significant gains in two standard benchmarks THUMOS14 and Charades. Our method improves more than 12% mAP over a weakly supervised baseline, outperforms other weakly supervised state-of-the-art methods and only 4% behind the state-of-the-art supervised method in THUMOS14 dataset for action detection. Similarly, our method is only 0.3% mAP behind a state-of-the-art supervised method on challenging Charades dataset for action localisation.
Abstract (translated)
检测视频中人类行为的时间范围是一个具有挑战性的计算机视觉问题，需要详细的人工监控，包括帧级标签。这种昂贵的注释过程限制了在有限数量的类别上部署动作检测器。我们提出了一种新的动作识别方法，称为wsgn，它可以从“弱监控”的视频级标签中学习检测动作。WSGN学习利用视频特定和数据集范围的统计信息来预测每个帧与动作类别的相关性。我们表明，本地和全球渠道的结合导致了两个标准基准周四14和字谜游戏的显著收益。我们的方法在弱监督基线上改进了超过12%的MAP，优于其他弱监督最先进的方法，在Thumos14动作检测数据集中仅落后于最先进的监督方法4%。同样地，我们的方法也只是落后于最先进的关于挑战性的字谜数据集的监控方法的0.3%。
URL
https://arxiv.org/abs/1904.07774
