Abstract
Text-to-image diffusion models excel at generating diverse portraits, but lack intuitive shadow control. Existing editing approaches, as post-processing, struggle to offer effective manipulation across diverse styles. Additionally, these methods either rely on expensive real-world light-stage data collection or require extensive computational resources for training. To address these limitations, we introduce Shadow Director, a method that extracts and manipulates hidden shadow attributes within well-trained diffusion models. Our approach uses a small estimation network that requires only a few thousand synthetic images and hours of training-no costly real-world light-stage data needed. Shadow Director enables parametric and intuitive control over shadow shape, placement, and intensity during portrait generation while preserving artistic integrity and identity across diverse styles. Despite training only on synthetic data built on real-world identities, it generalizes effectively to generated portraits with diverse styles, making it a more accessible and resource-friendly solution.
Abstract (translated)
文本到图像的扩散模型在生成多样的肖像方面表现出色，但缺乏直观的阴影控制。现有的编辑方法作为后处理手段，在处理不同风格时难以提供有效的操作。此外，这些方法要么依赖于昂贵的真实世界光舞台数据收集，要么需要大量的计算资源进行训练。为了解决这些问题，我们介绍了Shadow Director方法，该方法可以从已经训练好的扩散模型中提取并操纵隐藏的阴影属性。我们的方法使用一个小型估计网络，只需要几千张合成图像和几个小时的训练时间——无需昂贵的真实世界光舞台数据。 Shadow Director在生成肖像时提供了参数化且直观的阴影形状、位置及强度控制，并能在保持艺术完整性和身份一致性的前提下应用于各种风格中。尽管仅基于真实世界的身份构建并经过少量合成数据训练，它仍然能够有效地推广到具有多样风格的生成肖像上，使其成为一个更易于使用和资源友好的解决方案。
URL
https://arxiv.org/abs/2503.21943
