Abstract
Recent advancements in neural representations, such as Neural Radiance Fields and 3D Gaussian Splatting, have increased interest in applying style transfer to 3D scenes. While existing methods can transfer style patterns onto 3D-consistent neural representations, they struggle to effectively extract and transfer high-level style semantics from the reference style image. Additionally, the stylized results often lack structural clarity and separation, making it difficult to distinguish between different instances or objects within the 3D scene. To address these limitations, we propose a novel 3D style transfer pipeline that effectively integrates prior knowledge from pretrained 2D diffusion models. Our pipeline consists of two key stages: First, we leverage diffusion priors to generate stylized renderings of key viewpoints. Then, we transfer the stylized key views onto the 3D representation. This process incorporates two innovative designs. The first is cross-view style alignment, which inserts cross-view attention into the last upsampling block of the UNet, allowing feature interactions across multiple key views. This ensures that the diffusion model generates stylized key views that maintain both style fidelity and instance-level consistency. The second is instance-level style transfer, which effectively leverages instance-level consistency across stylized key views and transfers it onto the 3D representation. This results in a more structured, visually coherent, and artistically enriched stylization. Extensive qualitative and quantitative experiments demonstrate that our 3D style transfer pipeline significantly outperforms state-of-the-art methods across a wide range of scenes, from forward-facing to challenging 360-degree environments. Visit our project page this https URL for immersive visualization.
Abstract (translated)
近期在神经表示领域的进展，如神经辐射场（Neural Radiance Fields）和3D高斯点阵（3D Gaussian Splatting），引发了将风格迁移应用于三维场景的兴趣。尽管现有的方法能够将样式图案转移到具有三维一致性的神经表示上，但它们难以有效提取并转移参考样式图像中的高级别样式语义信息。此外，经过样式的处理结果往往缺乏结构清晰度和分离性，使得在三维场景中区分不同的实例或物体变得困难。 为了解决这些问题，我们提出了一种新的3D风格迁移流水线，该管道能够有效地整合预训练的2D扩散模型中的先验知识。我们的流水线包含两个关键阶段：首先，利用扩散先验生成主要视角下的样式化渲染；然后将这些样式化的主视图转移到三维表示上。这一过程包含了两项创新设计。 第一项是跨视角风格对齐（Cross-view style alignment），它在U-Net的最后一个上采样块中插入了跨视角注意力机制，允许多个关键视点之间的特征交互。这确保扩散模型生成的样式化主视图既保持样式保真度又维护实例级别的连贯性。 第二项是实例级风格迁移（Instance-level style transfer），它有效地利用了不同样式化主视图之间的一致性，并将其转移至三维表示上，从而产生结构更清晰、视觉一致性更强且艺术效果更加丰富的样化处理结果。 通过广泛的定性和定量实验证明，我们的3D风格迁移流水线在从正面对准到挑战性的全方位环境等各种场景中都显著优于当前最先进的方法。欢迎访问我们项目的网页 [这个链接]（请将此链接替换为实际项目页面的URL）以进行沉浸式可视化体验。
URL
https://arxiv.org/abs/2509.04379
