Abstract
By leveraging the blur-noise trade-off, imaging with non-uniform exposures largely extends the image acquisition flexibility in harsh environments. However, the limitation of conventional cameras in perceiving intra-frame dynamic information prevents existing methods from being implemented in the real-world frame acquisition for real-time adaptive camera shutter control. To address this challenge, we propose a novel Neuromorphic Shutter Control (NSC) system to avoid motion blurs and alleviate instant noises, where the extremely low latency of events is leveraged to monitor the real-time motion and facilitate the scene-adaptive exposure. Furthermore, to stabilize the inconsistent Signal-to-Noise Ratio (SNR) caused by the non-uniform exposure times, we propose an event-based image denoising network within a self-supervised learning paradigm, i.e., SEID, exploring the statistics of image noises and inter-frame motion information of events to obtain artificial supervision signals for high-quality imaging in real-world scenes. To illustrate the effectiveness of the proposed NSC, we implement it in hardware by building a hybrid-camera imaging prototype system, with which we collect a real-world dataset containing well-synchronized frames and events in diverse scenarios with different target scenes and motion patterns. Experiments on the synthetic and real-world datasets demonstrate the superiority of our method over state-of-the-art approaches.
Abstract (translated)
通过利用模糊噪声的权衡，非均匀曝光的成像在恶劣环境中大大扩展了图像采集的灵活性。然而，传统相机在感知帧内动态信息方面的局限性限制了现有方法在现实场景中实时自适应相机快门控制中的应用。为解决这个问题，我们提出了一个新的神经元仿真的快门控制（NSC）系统，以避免运动模糊并减轻即时噪声，其中利用事件极低延迟来监测实时运动并促进场景自适应曝光。此外，为了稳定由非均匀曝光时间引起的不一致信号-噪声比（SNR），我们提出了一个基于自监督学习范式的图像去噪网络，即SEID，通过分析事件图像噪声和事件间运动信息来获得高质量成像在现实场景中的的人工监督信号。为了说明所提出的NSC的有效性，我们在硬件上通过构建一个混合式相机成像原型系统，收集包含不同场景和运动模式下同步帧和事件的现实世界数据集。在合成和现实世界数据集上的实验证明，我们的方法相对于最先进的解决方案具有优越性。
URL
https://arxiv.org/abs/2404.13972
