Abstract
3D Gaussian Splatting (3DGS) excels in photorealistic scene reconstruction but struggles with stylized scenarios (e.g., cartoons, games) due to fragmented textures, semantic misalignment, and limited adaptability to abstract aesthetics. We propose StyleMe3D, a holistic framework for 3D GS style transfer that integrates multi-modal style conditioning, multi-level semantic alignment, and perceptual quality enhancement. Our key insights include: (1) optimizing only RGB attributes preserves geometric integrity during stylization; (2) disentangling low-, medium-, and high-level semantics is critical for coherent style transfer; (3) scalability across isolated objects and complex scenes is essential for practical deployment. StyleMe3D introduces four novel components: Dynamic Style Score Distillation (DSSD), leveraging Stable Diffusion's latent space for semantic alignment; Contrastive Style Descriptor (CSD) for localized, content-aware texture transfer; Simultaneously Optimized Scale (SOS) to decouple style details and structural coherence; and 3D Gaussian Quality Assessment (3DG-QA), a differentiable aesthetic prior trained on human-rated data to suppress artifacts and enhance visual harmony. Evaluated on NeRF synthetic dataset (objects) and tandt db (scenes) datasets, StyleMe3D outperforms state-of-the-art methods in preserving geometric details (e.g., carvings on sculptures) and ensuring stylistic consistency across scenes (e.g., coherent lighting in landscapes), while maintaining real-time rendering. This work bridges photorealistic 3D GS and artistic stylization, unlocking applications in gaming, virtual worlds, and digital art.
Abstract (translated)
3D高斯点云（3DGS）在逼真的场景重建方面表现出色，但在处理风格化场景（例如卡通、游戏等）时遇到了挑战。由于纹理碎片化、语义不匹配和对抽象美学适应性有限的原因，它在这方面表现不佳。为此，我们提出了StyleMe3D——一个用于3D GS风格转换的综合性框架，该框架集成了多模态风格条件、多层次语义对齐以及感知质量增强。我们的关键见解包括： 1. 仅优化RGB属性可以在风格化过程中保持几何完整性。 2. 解耦低级、中级和高级语义对于连贯的风格转移至关重要。 3. 对孤立对象和复杂场景的扩展性是实际部署中的必要条件。 StyleMe3D引入了四个新颖组件：动态风格分数蒸馏（Dynamic Style Score Distillation，DSSD），利用Stable Diffusion的潜在空间进行语义对齐；对比风格描述符（Contrastive Style Descriptor，CSD），用于局部化、内容感知的纹理转移；同时优化尺度（Simultaneously Optimized Scale，SOS），以解耦样式细节和结构一致性；以及3D高斯质量评估（3DG-QA），这是一种基于人类评分数据训练而成的可微美学术前项，能够抑制伪影并增强视觉和谐。 在NeRF合成数据集（对象）和tandt db（场景）数据集中进行评估后，StyleMe3D在保留几何细节（例如雕塑上的雕刻）以及确保跨场景风格一致性（例如景观中的连贯照明）方面优于现有最先进的方法，并且能够保持实时渲染。这项工作将逼真的3D GS与艺术风格化结合在一起，解锁了游戏、虚拟世界和数字艺术领域的应用。
URL
https://arxiv.org/abs/2504.15281
