Abstract
Over the past two decades, mobile imaging has experienced a profound transformation, with cell phones rapidly eclipsing all other forms of digital photography in popularity. Today's cell phones are equipped with a diverse range of imaging technologies - laser depth ranging, multi-focal camera arrays, and split-pixel sensors - alongside non-visual sensors such as gyroscopes, accelerometers, and magnetometers. This, combined with on-board integrated chips for image and signal processing, makes the cell phone a versatile pocket-sized computational imaging platform. Parallel to this, we have seen in recent years how neural fields - small neural networks trained to map continuous spatial input coordinates to output signals - enable the reconstruction of complex scenes without explicit data representations such as pixel arrays or point clouds. In this thesis, I demonstrate how carefully designed neural field models can compactly represent complex geometry and lighting effects. Enabling applications such as depth estimation, layer separation, and image stitching directly from collected in-the-wild mobile photography data. These methods outperform state-of-the-art approaches without relying on complex pre-processing steps, labeled ground truth data, or machine learning priors. Instead, they leverage well-constructed, self-regularized models that tackle challenging inverse problems through stochastic gradient descent, fitting directly to raw measurements from a smartphone.
Abstract (translated)
在过去二十年里，移动成像经历了深刻的变革，手机迅速超越了所有其他形式的数字摄影，在流行度上占据主导地位。今天的手机配备了各种成像技术——包括激光深度测距、多焦点相机阵列和分隔像素传感器——以及非视觉传感器，如陀螺仪、加速度计和磁力计。结合内置的图像和信号处理集成电路芯片，使手机成为一种多功能的手持式计算成像平台。 与此同时，在近年来我们见证了神经场（小型经过训练以将连续空间输入坐标映射到输出信号的小型神经网络）的发展，它们能够无需显式的数据表示（如像素阵列或点云）来重建复杂场景。在我的论文中，我展示了精心设计的神经场模型如何紧凑地代表复杂的几何结构和光照效果，并支持从收集的实际移动摄影数据直接进行深度估计、图层分离和图像拼接等应用。 这些方法在不依赖复杂的预处理步骤、标记的真实数据或机器学习先验的情况下超越了现有最佳的方法。相反，它们利用精心构造的、自我调节的模型通过随机梯度下降直接拟合智能手机采集的原始测量值来解决具有挑战性的逆向问题。
URL
https://arxiv.org/abs/2508.05907
