Abstract
Snapshot compressive imaging (SCI) is a promising technique for capturing high-speed video at low bandwidth and low power, typically by compressing multiple frames into a single measurement. However, similar to traditional CMOS image sensor based imaging systems, SCI also faces challenges in low-lighting photon-limited and low-signal-to-noise-ratio image conditions. In this paper, we propose a novel Compressive Denoising Autoencoder (CompDAE) using the STFormer architecture as the backbone, to explicitly model noise characteristics and provide computer vision functionalities such as edge detection and depth estimation directly from compressed sensing measurements, while accounting for realistic low-photon conditions. We evaluate the effectiveness of CompDAE across various datasets and demonstrated significant improvements in task performance compared to conventional RGB-based methods. In the case of ultra-low-lighting (APC $\leq$ 20) while conventional methods failed, the proposed algorithm can still maintain competitive performance.
Abstract (translated)
快照压缩成像（SCI）是一种有前景的技术，可在低带宽和低功耗条件下捕捉高速视频，通常通过将多个帧压缩为单个测量值来实现。然而，与传统的基于CMOS图像传感器的成像系统类似，SCI在光线不足、光子数量有限以及信噪比低的图像条件下也面临挑战。本文中，我们提出了一种使用STFormer架构作为骨干的新颖压缩去噪自动编码器（CompDAE），该编码器能够明确地建模噪声特征，并直接从压缩感知测量值提供计算机视觉功能，例如边缘检测和深度估计，在同时考虑现实中的低光子条件时也是如此。我们在多个数据集上评估了CompDAE的有效性，并展示了与传统基于RGB的方法相比在任务性能上的显著改进。即使在极低光照（APC ≤ 20）的情况下，常规方法失效，所提出的算法仍能保持竞争力的性能。
URL
https://arxiv.org/abs/2501.15122
