Abstract
3D Gaussian Splatting (3DGS) has emerged as a promising representation for photorealistic rendering of 3D scenes. However, its high storage requirements pose significant challenges for practical applications. We observe that Gaussians exhibit distinct roles and characteristics that are analogous to traditional artistic techniques -- Like how artists first sketch outlines before filling in broader areas with color, some Gaussians capture high-frequency features like edges and contours; While other Gaussians represent broader, smoother regions, that are analogous to broader brush strokes that add volume and depth to a painting. Based on this observation, we propose a novel hybrid representation that categorizes Gaussians into (i) Sketch Gaussians, which define scene boundaries, and (ii) Patch Gaussians, which cover smooth regions. Sketch Gaussians are efficiently encoded using parametric models, leveraging their geometric coherence, while Patch Gaussians undergo optimized pruning, retraining, and vector quantization to maintain volumetric consistency and storage efficiency. Our comprehensive evaluation across diverse indoor and outdoor scenes demonstrates that this structure-aware approach achieves up to 32.62% improvement in PSNR, 19.12% in SSIM, and 45.41% in LPIPS at equivalent model sizes, and correspondingly, for an indoor scene, our model maintains the visual quality with 2.3% of the original model size.
Abstract (translated)
3D高斯点阵（3DGS）作为一种具有前景的表示方法，被应用于三维场景的真实感渲染。然而，其高昂的数据存储需求对实际应用构成了重大挑战。我们观察到，高斯函数在角色和特性上与传统的艺术技巧相似——就像艺术家先勾勒轮廓线再用更宽的笔触填充区域一样，一些高斯函数捕捉了高频特征（如边缘和轮廓），而其他高斯函数则代表更广泛、平滑的区域，类似于为一幅画增加体积感和深度的宽刷子笔触。基于这一观察，我们提出了一种新的混合表示方法，将高斯函数分为两类：(i) 轮廓高斯（Sketch Gaussians），定义场景边界；以及(ii) 补丁高斯（Patch Gaussians），覆盖平滑区域。轮廓高斯通过参数模型进行高效编码，利用其几何一致性，而补丁高斯则经过优化剪枝、重新训练和向量量化处理以保持体积一致性和存储效率。 我们对多种室内及室外场景进行了全面评估，结果表明这种结构感知的方法在等效模型大小下，PSNR（峰值信噪比）提高了多达32.62%，SSIM（结构相似度指数）提升了19.12%，LPIPS（相对图片感知质量评分）减少了45.41%。对于一个室内场景来说，我们的模型可以在仅占原始模型大小的2.3%的情况下保持视觉效果的质量。
URL
https://arxiv.org/abs/2501.13045
