Abstract
We focus on a very challenging task: imaging at nighttime dynamic scenes. Most previous methods rely on the low-light enhancement of a conventional RGB camera. However, they would inevitably face a dilemma between the long exposure time of nighttime and the motion blur of dynamic scenes. Event cameras react to dynamic changes with higher temporal resolution (microsecond) and higher dynamic range (120dB), offering an alternative solution. In this work, we present a novel nighttime dynamic imaging method with an event camera. Specifically, we discover that the event at nighttime exhibits temporal trailing characteristics and spatial non-stationary distribution. Consequently, we propose a nighttime event reconstruction network (NER-Net) which mainly includes a learnable event timestamps calibration module (LETC) to align the temporal trailing events and a non-uniform illumination aware module (NIAM) to stabilize the spatiotemporal distribution of events. Moreover, we construct a paired real low-light event dataset (RLED) through a co-axial imaging system, including 64,200 spatially and temporally aligned image GTs and low-light events. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art methods in terms of visual quality and generalization ability on real-world nighttime datasets. The project are available at: this https URL.
Abstract (translated)
我们将注意力集中在一个非常具有挑战性的任务上：夜间动态场景的成像。大多数以前的方法依赖于传统RGB摄像机 low-light 增强。然而，它们会不可避免地面临夜间长曝光时间和动态场景运动模糊之间的困境。事件相机对动态变化具有更高的时间分辨率（微秒）和更高的动态范围（120dB），提供了另一种解决方案。在这项工作中，我们提出了一个事件相机驱动的夜间动态成像方法。具体来说，我们发现夜间事件表现出时间拖尾特征和空间非平稳分布。因此，我们提出了一个基于事件时钟的夜间事件重建网络（NER-Net），主要包括可学习的事件时间戳校准模块（LETC）和一个非均匀光照感知模块（NIAM），用于稳定事件的空间和时间分布。此外，我们还构建了一个通过轴向成像系统构建的成对低光事件数据集（RLED），包括64,200个空间和时间对齐的图像GT和低光事件。大量实验证明，与最先进的method相比，所提出的方法在现实世界的夜间数据集上的视觉质量和泛化能力都具有优势。该项目 available at： this https URL.
URL
https://arxiv.org/abs/2404.11884
