Abstract
In this paper, a novel video classification methodology is presented that aims to recognize different categories of third-person videos efficiently. The idea here is to break the 3-dimensional video input to 2D in spatial plus 1D in temporal dimension. Firstly, optical flow images are described by well pre-trained networks to process the 2D spatial frames. Then, motion in the video is kept tracked by aligning the optical flow elements over time which can be seen as multi-channel time series. The main focus of the proposed method is to classify the resulted time series efficiently. Towards this, the idea is to let the machine learn temporal features along time dimension. This is done by training a multi-channel one dimensional Convolutional Neural Network (1D-CNN). Due to the fact that CNNs represent the input data hierarchically, high level features are obtained by further processing of features in the lower level layers. As a result, long-term temporal features in time series are extracted from short-term ones. These long-term temporal features are the ones which play the key role in recognizing the ongoing action. Besides, the superiority of the proposed method over most of the deep-learning based approaches is that we try to learn representative temporal features across only one dimension. This, reduces the number of learning parameters significantly. Hence our method would be trainable on even smaller datasets. It is illustrated that the proposed method could reach state-of-the-art results on two public datasets UCF11 and jHMDB and competitive results on HMDB51.
Abstract (translated)
URL
https://arxiv.org/abs/1802.06724
