Abstract
Traditional cameras face a trade-off between low-light performance and high-speed imaging: longer exposure times to capture sufficient light results in motion blur, whereas shorter exposures result in Poisson-corrupted noisy images. While burst photography techniques help mitigate this tradeoff, conventional cameras are fundamentally limited in their sensor noise characteristics. Event cameras and single-photon avalanche diode (SPAD) sensors have emerged as promising alternatives to conventional cameras due to their desirable properties. SPADs are capable of single-photon sensitivity with microsecond temporal resolution, and event cameras can measure brightness changes up to 1 MHz with low bandwidth requirements. We show that these properties are complementary, and can help achieve low-light, high-speed image reconstruction with low bandwidth requirements. We introduce a sensor fusion framework to combine SPADs with event cameras to improves the reconstruction of high-speed, low-light scenes while reducing the high bandwidth cost associated with using every SPAD frame. Our evaluation, on both synthetic and real sensor data, demonstrates significant enhancements ( > 5 dB PSNR) in reconstructing low-light scenes at high temporal resolution (100 kHz) compared to conventional cameras. Event-SPAD fusion shows great promise for real-world applications, such as robotics or medical imaging.
Abstract (translated)
传统相机在低光性能和高帧速成像之间存在权衡：较长的曝光时间可以捕捉到足够的照明，但较短的曝光时间会导致运动模糊，而较长的曝光时间会导致Poisson噪声污染的图像。尽管快速摄影技术有助于减轻这种权衡，但传统相机的传感器噪声特性根本有限。事件相机和单光子 avalanche 器件（SPAD）传感器已成为传统相机的有前景的替代品，因为它们具有所需的特性。SPAD具有微秒级的时间分辨率，能够实现单光子灵敏度，而事件相机可以在低带宽条件下测量亮度变化高达1 MHz。我们证明了这些特性是互补的，可以帮助实现低光，高速图像重建，同时降低使用每个SPAD帧的高带宽成本。我们在合成和真实传感器数据上进行的评估表明，与传统相机相比，在高速低光场景中重建低光图像的性能明显增强（>5 dB PSNR）。事件-SPAD融合在现实应用领域（如机器人或医学成像）具有巨大的潜力。
URL
https://arxiv.org/abs/2404.11511
