Abstract
Video Compressed Sensing (VCS) aims to reconstruct multiple frames from one single captured measurement, thus achieving high-speed scene recording with a low-frame-rate sensor. Although there have been impressive advances in VCS recently, those state-of-the-art (SOTA) methods also significantly increase model complexity and suffer from poor generality and robustness, which means that those networks need to be retrained to accommodate the new system. Such limitations hinder the real-time imaging and practical deployment of models. In this work, we propose a Sampling-Priors-Augmented Deep Unfolding Network (SPA-DUN) for efficient and robust VCS reconstruction. Under the optimization-inspired deep unfolding framework, a lightweight and efficient U-net is exploited to downsize the model while improving overall performance. Moreover, the prior knowledge from the sampling model is utilized to dynamically modulate the network features to enable single SPA-DUN to handle arbitrary sampling settings, augmenting interpretability and generality. Extensive experiments on both simulation and real datasets demonstrate that SPA-DUN is not only applicable for various sampling settings with one single model but also achieves SOTA performance with incredible efficiency.
Abstract (translated)
视频压缩感知(VCS)的目标是从单个捕获测量中重构多个帧,从而实现低帧率传感器的高速度场景录制。尽管最近在VCS方面取得了令人印象深刻的进展,但这些先进的方法也显著增加了模型的复杂性并出现了 poor generality和Robustness 的问题,这意味着这些网络需要适应新的系统并进行训练。这些限制妨碍了实时成像和模型的实际部署。在本文中,我们提出了一种采样先验增强深度展开网络(SPA-DUN)来高效和稳健地 VCS 重构。在基于优化的深度展开框架下,利用轻量级且高效的 U-net 减小模型大小并提高整体性能。此外,从采样模型的先验知识动态地调节网络特征,使单个 SPA-DUN 能够处理任意采样设置,增加可解释性和一般性。在模拟和真实数据集上的广泛实验表明,SPA-DUN不仅可以适用于单个模型的各种采样设置,而且具有惊人的效率和 SOTA 性能。
URL
https://arxiv.org/abs/2307.07291
