Abstract
Video prediction aims to generate realistic future frames by learning dynamic visual patterns. One fundamental challenge is to deal with future uncertainty: How should a model behave when there are multiple correct, equally probable future? We propose an Appearance-Motion Conditional GAN to address this challenge. We provide appearance and motion information as conditions that specify how the future may look like, reducing the level of uncertainty. Our model consists of a generator, two discriminators taking charge of appearance and motion pathways, and a perceptual ranking module that encourages videos of similar conditions to look similar. To train our model, we develop a novel conditioning scheme that consists of different combinations of appearance and motion conditions. We evaluate our model using facial expression and human action datasets and report favorable results compared to existing methods.
Abstract (translated)
视频预测旨在通过学习动态视觉模式来生成逼真的未来帧。一个基本的挑战是应对未来的不确定性：当存在多个正确的，同样可能的未来时，模型应该如何表现？我们提出了一个Appearance-Motion有条件GAN来应对这一挑战。我们提供外观和运动信息作为指定未来如何的条件，降低不确定性水平。我们的模型包括一个生成器，两个负责外观和运动路径的鉴别器，以及一个感知排名模块，鼓励类似条件的视频看起来相似。为了训练我们的模型，我们开发了一种新颖的调节方案，该方案由外观和运动条件的不同组合组成。我们使用面部表情和人类活动数据集评估我们的模型，并报告与现有方法相比的有利结果。
URL
https://arxiv.org/abs/1807.02635
