Abstract
Recently, Gaussian Splatting methods have emerged as a desirable substitute for prior Radiance Field methods for novel-view synthesis of scenes captured with multi-view images or videos. In this work, we propose a novel extension to 4D Gaussian Splatting for dynamic scenes. Drawing on ideas from residual learning, we hierarchically decompose the dynamic scene into a "video-segment-frame" structure, with segments dynamically adjusted by optical flow. Then, instead of directly predicting the time-dependent signals, we model the signal as the sum of video-constant values, segment-constant values, and frame-specific residuals, as inspired by the success of residual learning. This approach allows more flexible models that adapt to highly variable scenes. We demonstrate state-of-the-art visual quality and real-time rendering on several established datasets, with the greatest improvements on complex scenes with large movements, occlusions, and fine details, where current methods degrade most.
Abstract (translated)
最近，高斯点置法（Gaussian Splatting）作为一种新型方法，在使用多视角图像或视频捕获的场景的新视图合成中，成为之前辐射场（Radiance Field）方法的理想替代方案。在本项工作中，我们提出了一种针对动态场景扩展4D高斯点置法的创新方法。借鉴残差学习的思想，我们将动态场景层次化地分解成“视频片段帧”结构，并通过光流进行动态调整。然后，我们不像直接预测时间依赖信号那样处理问题，而是将信号建模为视频常量值、片段常量值和特定帧残差的总和，这受到残差学习成功的启发。这种方法使得模型能够更灵活地适应变化多端的场景。我们在几个已建立的数据集上展示了最先进的视觉质量和实时渲染性能，并且在包含大量移动、遮挡以及细节复杂场景的情况下改进最大，这是当前方法表现最差的地方。
URL
https://arxiv.org/abs/2505.18306
