Abstract
Deep network-based image and video Compressive Sensing(CS) has attracted increasing attentions in recent years. However, in the existing deep network-based CS methods, a simple stacked convolutional network is usually adopted, which not only weakens the perception of rich contextual prior knowledge, but also limits the exploration of the correlations between temporal video frames. In this paper, we propose a novel Hierarchical InTeractive Video CS Reconstruction Network(HIT-VCSNet), which can cooperatively exploit the deep priors in both spatial and temporal domains to improve the reconstruction quality. Specifically, in the spatial domain, a novel hierarchical structure is designed, which can hierarchically extract deep features from keyframes and non-keyframes. In the temporal domain, a novel hierarchical interaction mechanism is proposed, which can cooperatively learn the correlations among different frames in the multiscale space. Extensive experiments manifest that the proposed HIT-VCSNet outperforms the existing state-of-the-art video and image CS methods in a large margin.
Abstract (translated)
深度学习图像和视频压缩感知(CS)近年来日益受到关注。然而，在现有的深度学习CS方法中，通常采用简单的叠加卷积神经网络，这不仅削弱了丰富的上下文先验知识的感受度，而且也限制了对时间帧之间相关性的探索。在本文中，我们提出了一种新的Hierarchical InTeractive Video CS Reconstruction Network(HIT-VCSNet)，它可以合作利用空间和时间域中的深层先验知识，以提高重建质量。具体来说，在空间域中，我们设计了一种 novel Hierarchical 结构，可以从关键帧和非关键帧中Hierarchically 提取深层特征。在时间域中，我们提出了一种 novel Hierarchical 交互机制，可以在多尺度空间中合作学习不同帧之间的相关性。广泛的实验表明，提出的HIT-VCSNet在显著优于现有的先进的视频和图像CS方法。
URL
https://arxiv.org/abs/2304.07473
