Abstract
Scene stylization extends the work of neural style transfer to three spatial dimensions. A vital challenge in this problem is to maintain the uniformity of the stylized appearance across a multi-view setting. A vast majority of the previous works achieve this by optimizing the scene with a specific style image. In contrast, we propose a novel architecture trained on a collection of style images, that at test time produces high quality stylized novel views. Our work builds up on the framework of 3D Gaussian splatting. For a given scene, we take the pretrained Gaussians and process them using a multi resolution hash grid and a tiny MLP to obtain the conditional stylised views. The explicit nature of 3D Gaussians give us inherent advantages over NeRF-based methods including geometric consistency, along with having a fast training and rendering regime. This enables our method to be useful for vast practical use cases such as in augmented or virtual reality applications. Through our experiments, we show our methods achieve state-of-the-art performance with superior visual quality on various indoor and outdoor real-world data.
Abstract (translated)
场景风格化扩展了神经风格迁移在三维空间的工作。这个问题中的一个关键挑战是保持风格化外观在多视角设置中的统一性。大部分之前的工作通过优化特定风格图像的场景来实现这一点。相比之下，我们提出了一个基于样式图像的集合训练的新模型，在测试时产生高质量的风格化新视图。我们的工作基于3D高斯分层的框架。对于给定的场景，我们使用预训练的高斯核并对其进行多分辨率哈希网格处理和微小的MLP处理，以获得条件风格化视图。3D高斯核的显式性质使我们比基于NeRF的方法具有更强的几何一致性，并具有快速训练和渲染模式。这使我们能够为诸如增强现实和虚拟现实等广泛应用场景提供有用的方法。通过我们的实验，我们证明了我们的方法在各种室内和室外现实数据上实现了最先进的性能，具有卓越的视觉质量。
URL
https://arxiv.org/abs/2403.08498
