Abstract
Transferring image-based object detectors to domain of videos remains a challenging problem. Previous efforts mostly exploit optical flow to propagate features across frames, aiming to achieve a good trade-off between performance and computational complexity. However, introducing an extra model to estimate optical flow would significantly increase the overall model size. The gap between optical flow and high-level features can hinder it from establishing the spatial correspondence accurately. Instead of relying on optical flow, this paper proposes a novel module called Progressive Sparse Local Attention (PSLA), which establishes the spatial correspondence between features across frames in a local region with progressive sparse strides and uses the correspondence to propagate features. Based on PSLA, Recursive Feature Updating (RFU) and Dense feature Transforming (DFT) are introduced to model temporal appearance and enrich feature representation respectively. Finally, a novel framework for video object detection is proposed. Experiments on ImageNet VID are conducted. Our framework achieves a state-of-the-art speed-accuracy trade-off with significantly reduced model capacity.
Abstract (translated)
将基于图像的目标探测器传输到视频领域仍然是一个具有挑战性的问题。以前的工作主要是利用光流在帧之间传播特性，目的是在性能和计算复杂性之间实现良好的平衡。然而，引入一个额外的模型来估计光流量会显著增加模型的整体尺寸。光流与高阶特征之间的间隙会阻碍其准确建立空间对应关系。本文提出了一种新的渐进稀疏局部注意（PSLA）模块，它不依赖于光流，而是在一个渐进稀疏步幅的局部区域内建立跨帧特征之间的空间对应关系，并利用对应关系来传播特征。基于PSLA，引入递归特征更新（RFU）和密集特征变换（DFT）分别对时间外观进行建模，丰富了特征表示。最后，提出了一种新的视频目标检测框架。对图像网络视频进行了实验研究。我们的框架实现了最先进的速度精度权衡，大大降低了模型容量。
URL
https://arxiv.org/abs/1903.09126
