Abstract
Recent advances in text-driven 3D scene editing and stylization, which leverage the powerful capabilities of 2D generative models, have demonstrated promising outcomes. However, challenges remain in ensuring high-quality stylization and view consistency simultaneously. Moreover, applying style consistently to different regions or objects in the scene with semantic correspondence is a challenging task. To address these limitations, we introduce techniques that enhance the quality of 3D stylization while maintaining view consistency and providing optional region-controlled style transfer. Our method achieves stylization by re-training an initial 3D representation using stylized multi-view 2D images of the source views. Therefore, ensuring both style consistency and view consistency of stylized multi-view images is crucial. We achieve this by extending the style-aligned depth-conditioned view generation framework, replacing the fully shared attention mechanism with a single reference-based attention-sharing mechanism, which effectively aligns style across different viewpoints. Additionally, inspired by recent 3D inpainting methods, we utilize a grid of multiple depth maps as a single-image reference to further strengthen view consistency among stylized images. Finally, we propose Multi-Region Importance-Weighted Sliced Wasserstein Distance Loss, allowing styles to be applied to distinct image regions using segmentation masks from off-the-shelf models. We demonstrate that this optional feature enhances the faithfulness of style transfer and enables the mixing of different styles across distinct regions of the scene. Experimental evaluations, both qualitative and quantitative, demonstrate that our pipeline effectively improves the results of text-driven 3D stylization.
Abstract (translated)
近期在文本驱动的三维场景编辑和风格化方面取得的进步，利用了二维生成模型的强大能力，并展示了有希望的结果。然而，在确保高质量的风格化的同时保持视角一致性仍然存在挑战。此外，将样式一致地应用于具有语义对应的不同区域或物体是一项艰巨的任务。为了解决这些限制，我们引入了一系列技术来增强3D风格化的质量，同时保持视角一致性并提供可选的区域控制式样传输。我们的方法通过使用源视图的风格化多视角2D图像重新训练初始3D表示来实现样式化。因此，确保风格化多视角图像的风格一致性和视角一致性至关重要。 为了达到这一目标，我们扩展了与风格对齐的深度条件视角生成框架，并用单一参考基础注意力共享机制替换了完全共享注意力机制，这有效地将不同视点上的风格进行了对齐。此外，受最近的3D修复方法启发，我们利用多个深度图网格作为单张图像的参考，进一步加强了风格化图像之间的视角一致性。 最后，我们提出了一种多区域重要性加权切片Wasserstein距离损失，通过使用现成模型生成的分割掩码，允许不同的样式应用于独特的图像区域。证明这一可选特性增强了样式的传输保真度，并使不同区域混合不同样式成为可能。实验评估（定性和定量）表明，我们的流程有效地提高了文本驱动3D风格化的结果。 翻译总结了原文献的主要贡献和创新点，展示了如何通过一系列技术改进来解决当前三维场景编辑和风格化中的主要挑战，从而提升图像的质量、一致性和灵活性。
URL
https://arxiv.org/abs/2509.05285
