Abstract
Generalized zero-shot action recognition is a challenging problem, where the task is to recognize new action categories that are unavailable during the training stage, in addition to the seen action categories. Existing approaches suffer from the inherent bias of the learned classifier towards the seen action categories. As a consequence, unseen category samples are incorrectly classified as belonging to one of the seen action categories. In this paper, we set out to tackle this issue by arguing for a separate treatment of seen and unseen action categories in generalized zero-shot action recognition. We introduce an out-of-distribution detector that determines whether the video features belong to a seen or unseen action category. To train our out-of-distribution detector, video features for unseen action categories are synthesized using generative adversarial networks trained on seen action category features. To the best of our knowledge, we are the first to propose an out-of-distribution detector based GZSL framework for action recognition in videos. Experiments are performed on three action recognition datasets: Olympic Sports, HMDB51 and UCF101. For generalized zero-shot action recognition, our proposed approach outperforms the baseline (f-CLSWGAN) with absolute gains (in classification accuracy) of 7.0%, 3.4%, and 4.9%, respectively, on these datasets.
Abstract (translated)
广义零镜头动作识别是一个具有挑战性的问题，其任务是识别训练阶段不可用的新动作类别，以及看到的动作类别。现有的方法受到所学分类器对所见操作类别的固有偏见的影响。因此，未看到的类别样本被错误地分类为属于所看到的操作类别之一。在本文中，我们试图通过对广义零镜头动作识别中可见和不可见的动作类别进行单独的处理来解决这个问题。我们引入了一个不分布检测器，该检测器确定视频特征是否属于可见或不可见的动作类别。为了训练我们的分布外探测器，我们使用生成的对抗性网络合成了看不见的动作类别的视频特征，这些网络训练了看不见的动作类别特征。据我们所知，我们首先提出了一种基于分布外检测器的gzsl视频动作识别框架。对奥林匹克运动、HMDB51和UCF101三个动作识别数据集进行了实验研究。对于广义零镜头动作识别，我们提出的方法优于基线（F-CLSWGAN），在这些数据集上的绝对增益（分类精度）分别为7.0%、3.4%和4.9%。
URL
https://arxiv.org/abs/1904.08703
