Abstract
Smartphone cameras have gained immense popularity with the adoption of high-resolution and high-dynamic range imaging. As a result, high-performance camera Image Signal Processors (ISPs) are crucial in generating high-quality images for the end user while keeping computational costs low. In this paper, we propose DRIFT (Deep Restoration, ISP Fusion, and Tone-mapping): an efficient AI mobile camera pipeline that generates high quality RGB images from hand-held raw captures. The first stage of DRIFT is a Multi-Frame Processing (MFP) network that is trained using a adversarial perceptual loss to perform multi-frame alignment, denoising, demosaicing, and super-resolution. Then, the output of DRIFT-MFP is processed by a novel deep-learning based tone-mapping (DRIFT-TM) solution that allows for tone tunability, ensures tone-consistency with a reference pipeline, and can be run efficiently for high-resolution images on a mobile device. We show qualitative and quantitative comparisons against state-of-the-art MFP and tone-mapping methods to demonstrate the effectiveness of our approach.
Abstract (translated)
智能手机摄像头凭借高分辨率与高动态范围成像技术的普及而广受欢迎。因此，高性能的图像信号处理器（ISP）对于在保持低计算成本的同时向终端用户生成高质量图像至关重要。本文提出了一种名为DRIFT（深度复原、ISP融合与色调映射）的高效AI移动端摄像头成像管线，能够从手持RAW格式拍摄中生成高质量RGB图像。DRIFT的第一阶段是一个多帧处理（MFP）网络，该网络通过对抗性感知损失进行训练，以实现多帧对齐、去噪、去马赛克和超分辨率处理。随后，DRIFT-MFP的输出会经过一种新型的基于深度学习的色调映射（DRIFT-TM）方案处理，该方案具备色调可调性，能确保与参考管线保持色调一致性，并且可在移动设备上高效运行高分辨率图像。我们通过定性与定量对比当前最先进的多帧处理与色调映射方法，验证了本方法的有效性。
URL
https://arxiv.org/abs/2604.03402
