Abstract
Predicting the near-future from an input video is a useful task for applications such as autonomous driving and robotics. While most previous works predict a single future, multiple futures with different behaviors can possibly occur. Moreover, if the predicted future is too short, it may not be fully usable by a human or other system. In this paper, we propose a novel method for future video prediction capable of generating multiple long-term futures. This makes the predictions more suitable for real applications. First, from an input human video, we generate sequences of future human poses as the image coordinates of their body-joints by adversarial learning. We generate multiple futures by inputting to the generator combinations of a latent code (to reflect various behaviors) and an attraction point (to reflect various trajectories). In addition, we generate long-term future human poses using a novel approach based on unidimensional convolutional neural networks. Last, we generate an output video based on the generated poses for visualization. We evaluate the generated future poses and videos using three criteria (i.e., realism, diversity and accuracy), and show that our proposed method outperforms other state-of-the-art works.
Abstract (translated)
从输入视频预测不久的将来，对于自动驾驶和机器人等应用程序来说是一项很有用的任务。虽然以前的大多数作品都预测一个未来，但可能会出现多个行为不同的未来。此外，如果预测的未来太短，人类或其他系统可能无法完全利用它。本文提出了一种能够产生多个长期期货的未来视频预测方法。这使得预测更适合实际应用。首先，从输入的人类视频中，我们通过对抗性学习生成未来人类姿势的序列，作为他们身体关节的图像坐标。我们通过输入潜在代码（以反映各种行为）和吸引点（以反映各种轨迹）的生成器组合来生成多个期货。此外，我们使用一种基于一维卷积神经网络的新方法来产生长期的未来人体姿势。最后，我们根据生成的姿势生成一个输出视频进行可视化。我们使用三个标准（即真实性、多样性和准确性）评估生成的未来姿势和视频，并表明我们的方法优于其他最先进的作品。
URL
https://arxiv.org/abs/1904.07538
