Abstract
Indoor lighting estimation from a single image or video remains a challenge due to its highly ill-posed nature, especially when the lighting condition of the scene varies spatially and temporally. We propose a method that estimates from an input video a continuous light field describing the spatiotemporally varying lighting of the scene. We leverage 2D diffusion priors for optimizing such light field represented as a MLP. To enable zero-shot generalization to in-the-wild scenes, we fine-tune a pre-trained image diffusion model to predict lighting at multiple locations by jointly inpainting multiple chrome balls as light probes. We evaluate our method on indoor lighting estimation from a single image or video and show superior performance over compared baselines. Most importantly, we highlight results on spatiotemporally consistent lighting estimation from in-the-wild videos, which is rarely demonstrated in previous works.
Abstract (translated)
从单张图像或视频中估计室内照明仍然是一项挑战，尤其是当场景的光照条件在空间和时间上变化时。我们提出了一种方法，可以从输入视频中估算出连续光场，描述场景中的时空光照变化。我们将二维扩散先验用于优化表示为多层感知机（MLP）的这种光场。为了实现对野外场景的零样本泛化，我们微调了一个预训练的图像扩散模型，通过联合绘制多个铬球作为照明探针来预测多个位置处的照明情况。 我们在单张图像或视频上的室内照明估计上评估了该方法，并显示其性能优于比较基准。最重要的是，我们强调了在野外视频中一致地估算时空照明的效果，这在以前的工作中很少被展示。
URL
https://arxiv.org/abs/2508.08384
