Abstract
We present a diffusion-based portrait shadow removal approach that can robustly produce high-fidelity results. Unlike previous methods, we cast shadow removal as diffusion-based inpainting. To this end, we first train a shadow-independent structure extraction network on a real-world portrait dataset with various synthetic lighting conditions, which allows to generate a shadow-independent structure map including facial details while excluding the unwanted shadow boundaries. The structure map is then used as condition to train a structure-guided inpainting diffusion model for removing shadows in a generative manner. Finally, to restore the fine-scale details (e.g., eyelashes, moles and spots) that may not be captured by the structure map, we take the gradients inside the shadow regions as guidance and train a detail restoration diffusion model to refine the shadow removal result. Extensive experiments on the benchmark datasets show that our method clearly outperforms existing methods, and is effective to avoid previously common issues such as facial identity tampering, shadow residual, color distortion, structure blurring, and loss of details. Our code is available at this https URL.
Abstract (translated)
我们提出了一种基于扩散的肖像阴影移除方法，能够稳健地产生高保真度的结果。与以往的方法不同，我们将阴影移除问题视为一种基于扩散的修复（inpainting）任务。为此，我们首先在一个包含多种合成光照条件的真实世界肖像数据集上训练了一个独立于阴影的结构提取网络，从而生成一个包括面部细节但排除不需要的阴影边界的独立于阴影的结构图。然后使用该结构图为指导来训练一个结构导向的修复扩散模型，以生成方式移除阴影。最后，为了恢复可能未被结构图捕捉到的细小特征（如睫毛、痣和斑点），我们采用阴影区域内的梯度作为引导，并训练一个细节恢复扩散模型对阴影移除结果进行精炼。 在基准数据集上的大量实验表明，我们的方法明显优于现有的方法，并能有效避免诸如面部身份篡改、阴影残留、颜色失真、结构模糊以及细节丢失等以往常见的问题。我们的代码可在以下链接获取：[此链接]（请将方括号中的内容替换为实际的URL）。
URL
https://arxiv.org/abs/2507.04692
