Abstract
Online reconstruction of dynamic scenes aims to learn from streaming multi-view inputs under low-latency constraints. The fast training and real-time rendering capabilities of 3D Gaussian Splatting have made on-the-fly reconstruction practically feasible, enabling online 4D reconstruction. However, existing online approaches, despite their efficiency and visual quality, fail to learn per-Gaussian motion that reflects true scene dynamics. Without explicit motion cues, appearance and motion are optimized solely under photometric loss, causing per-Gaussian motion to chase pixel residuals rather than true 3D motion. To address this, we propose MoRGS, an efficient online per-Gaussian motion reasoning framework that explicitly models per-Gaussian motion to improve 4D reconstruction quality. Specifically, we leverage optical flow on a sparse set of key views as lightweight motion cues that regularize per-Gaussian motion beyond photometric supervision. To compensate for the sparsity of flow supervision, we learn a per-Gaussian motion offset field that reconciles discrepancies between projected 3D motion and observed flow across views and time. In addition, we introduce a per-Gaussian motion confidence that separates dynamic from static Gaussians and weights Gaussian attribute residual updates, thereby suppressing redundant motion in static regions for better temporal consistency and accelerating the modeling of large motions. Extensive experiments demonstrate that MoRGS achieves state-of-the-art reconstruction quality and motion fidelity among online methods, while maintaining streamable performance.
Abstract (translated)
在线动态场景重建旨在低延迟约束下从流式多视角输入中学习。3D高斯泼溅（3D Gaussian Splatting）的快速训练与实时渲染能力使即时重建成为可能，从而支持在线4D重建。然而，现有在线方法尽管具备效率与视觉质量优势，却未能学习反映真实场景动态的每高斯运动。缺乏显式运动线索时，外观与运动仅通过光度损失进行优化，导致每高斯运动追逐像素残差而非真实3D运动。为解决此问题，我们提出MoRGS——一种高效的在线每高斯运动推理框架，通过显式建模每高斯运动来提升4D重建质量。具体而言，我们利用稀疏关键视角上的光流作为轻量级运动线索，以超越光度监督来约束每高斯运动。为弥补流监督的稀疏性，我们学习每高斯运动偏移场，以协调跨视角与时间上投影3D运动与观测流之间的差异。此外，我们引入每高斯运动置信度，区分动态与静态高斯，并对高斯属性残差更新进行加权，从而抑制静态区域的冗余运动以获得更佳时间一致性，并加速大运动建模。大量实验表明，MoRGS在在线方法中实现了最先进的重建质量与运动保真度，同时保持流式处理性能。
URL
https://arxiv.org/abs/2603.25042
