Abstract
Recent 3D-aware head generative models based on 3D Gaussian Splatting achieve real-time, photorealistic and view-consistent head synthesis. However, a fundamental limitation persists: the deep entanglement of illumination and intrinsic appearance prevents controllable relighting. Existing disentanglement methods rely on strong assumptions to enable weakly supervised learning, which restricts their capacity for complex illumination. To address this challenge, we introduce HeadLighter, a novel supervised framework that learns a physically plausible decomposition of appearance and illumination in head generative models. Specifically, we design a dual-branch architecture that separately models lighting-invariant head attributes and physically grounded rendering components. A progressive disentanglement training is employed to gradually inject head appearance priors into the generative architecture, supervised by multi-view images captured under controlled light conditions with a light stage setup. We further introduce a distillation strategy to generate high-quality normals for realistic rendering. Experiments demonstrate that our method preserves high-quality generation and real-time rendering, while simultaneously supporting explicit lighting and viewpoint editing. We will publicly release our code and dataset.
Abstract (translated)
近期基于3D高斯点阵的三维感知头部生成模型实现了实时、逼真且视角一致的头部合成。然而，这些模型仍存在一个基本限制：光照与内在外观之间的深度纠缠阻碍了可控重光照的能力。现有的脱耦方法依赖于强烈的假设来实现弱监督学习，这限制了它们处理复杂照明场景的能力。 为了应对这一挑战，我们引入了一种名为HeadLighter的新框架，该框架在头部生成模型中学习了一个物理上合理的外观和光照分解。具体来说，我们设计了一种双分支架构，分别对光照不变的头部属性和基于物理原理的渲染组件进行建模。通过使用多视角图像（这些图像是在具有灯光舞台设置的受控照明条件下捕获的）监督下逐步注入头部外观先验，采用渐进式脱耦训练方法。 此外，我们还引入了一种蒸馏策略来生成高质量的法线图以支持逼真的渲染。实验结果表明，我们的方法在保持高质量生成和实时渲染的同时，也能够支持明确的光照编辑和视角调整。我们将公开发布代码和数据集。
URL
https://arxiv.org/abs/2601.02103
