Abstract
This paper presents a new large-scale dataset for recognition and temporal localization of human actions collected from Web videos. We refer to it as HACS (Human Action Clips and Segments). We leverage consensus among visual classifiers to automatically mine candidate short clips from unlabeled videos, which are subsequently validated via manual verification. Annotated clips include both positive examples and hard negatives. The resulting dataset is dubbed HACS Clips. Through a separate process we also collect annotations defining action segment boundaries. This resulting dataset is called HACS Segments. Overall, HACS Clips consists of 1.55M annotated clips sampled from 504K untrimmed videos, and HACS Segments contains 139K action segments densely annotated in 50K untrimmed videos spanning 200 action categories. HACS Clips contains more labeled examples than any existing video benchmark. This renders our dataset an excellent source for spatiotemporal feature learning, as evidenced by our transfer learning experiments on three different target datasets where HACS Clips outperforms Kinetics and Sports1M as a pretraining benchmark, and yields the best published results to date. On HACS Segments, we evaluate state-of-the-art methods of action proposal generation and action localization, and highlight the new challenges posed by our dense and fine-grained temporal annotations.
Abstract (translated)
URL
https://arxiv.org/abs/1712.09374
