Abstract
Video summarization plays an important role in video understanding by selecting key frames/shots. Traditionally, it aims to find the most representative and diverse contents in a video as short summaries. Recently, a more generalized task, query-conditioned video summarization, has been introduced, which takes user queries into consideration to learn more user-oriented summaries. In this paper, we propose a query-conditioned three-player generative adversarial network to tackle this challenge. The generator learns the joint representation of the user query and the video content, and the discriminator takes three pairs of query-conditioned summaries as the input to discriminate the real summary from a generated and a random one. A three-player loss is introduced for joint training of the generator and the discriminator, which forces the generator to learn better summary results, and avoids the generation of random trivial summaries. Experiments on a recently proposed query-conditioned video summarization benchmark dataset show the efficiency and efficacy of our proposed method.
Abstract (translated)
视频摘要通过选择关键帧/镜头在视频理解中起重要作用。传统上，它旨在将视频中最具代表性和多样性的内容作为简短摘要。最近，引入了更通用的任务，查询条件的视频摘要，其考虑用户查询以学习更多面向用户的摘要。在本文中，我们提出了一个查询条件的三人生成对抗网络来应对这一挑战。生成器学习用户查询和视频内容的联合表示，并且鉴别器采用三对查询条件摘要作为输入以区分真实摘要与生成的和随机的摘要。引入了三个玩家的损失，用于联合训练发生器和鉴别器，这迫使发生器学习更好的总结结果，并避免产生随机的简单摘要。最近提出的查询条件视频摘要基准数据集的实验显示了我们提出的方法的效率和功效。
URL
https://arxiv.org/abs/1807.06677
