Abstract
The need for efficiently finding the video content a user wants is increasing because of the erupting of user-generated videos on the Web. Existing keyword-based or content-based video retrieval methods usually determine what occurs in a video but not when and where. In this paper, we make an answer to the question of when and where by formulating a new task, namely spatio-temporal video re-localization. Specifically, given a query video and a reference video, spatio-temporal video re-localization aims to localize tubelets in the reference video such that the tubelets semantically correspond to the query. To accurately localize the desired tubelets in the reference video, we propose a novel warp LSTM network, which propagates the spatio-temporal information for a long period and thereby captures the corresponding long-term dependencies. Another issue for spatio-temporal video re-localization is the lack of properly labeled video datasets. Therefore, we reorganize the videos in the AVA dataset to form a new dataset for spatio-temporal video re-localization research. Extensive experimental results show that the proposed model achieves superior performances over the designed baselines on the spatio-temporal video re-localization task.
Abstract (translated)
由于网络上出现了用户生成的视频，因此高效地查找用户想要的视频内容的需求越来越大。现有的基于关键字或基于内容的视频检索方法通常确定视频中发生了什么，但不确定何时何地发生。本文通过制定一个新的任务，即时空视频的重新定位，来回答“何时何地”的问题。具体地说，给定一个查询视频和一个参考视频，时空视频重新定位的目的是定位参考视频中的管元素，使管元素在语义上与查询相对应。为了准确定位参考视频中所需的管柱，我们提出了一种新的Warp LSTM网络，它可以长时间地传播时空信息，从而捕获相应的长期相关性。时空视频重新定位的另一个问题是缺乏正确标记的视频数据集。因此，我们重新组织AVA数据集中的视频，形成一个新的数据集，用于时空视频的重新定位研究。大量的实验结果表明，该模型在时空视频重定位任务中比设计的基线具有更好的性能。
URL
https://arxiv.org/abs/1905.03922
