Abstract
We introduce FLAG-4D, a novel framework for generating novel views of dynamic scenes by reconstructing how 3D Gaussian primitives evolve through space and time. Existing methods typically rely on a single Multilayer Perceptron (MLP) to model temporal deformations, and they often struggle to capture complex point motions and fine-grained dynamic details consistently over time, especially from sparse input views. Our approach, FLAG-4D, overcomes this by employing a dual-deformation network that dynamically warps a canonical set of 3D Gaussians over time into new positions and anisotropic shapes. This dual-deformation network consists of an Instantaneous Deformation Network (IDN) for modeling fine-grained, local deformations and a Global Motion Network (GMN) for capturing long-range dynamics, refined through mutual learning. To ensure these deformations are both accurate and temporally smooth, FLAG-4D incorporates dense motion features from a pretrained optical flow backbone. We fuse these motion cues from adjacent timeframes and use a deformation-guided attention mechanism to align this flow information with the current state of each evolving 3D Gaussian. Extensive experiments demonstrate that FLAG-4D achieves higher-fidelity and more temporally coherent reconstructions with finer detail preservation than state-of-the-art methods.
Abstract (translated)
我们介绍了一种新的框架FLAG-4D，用于通过重建三维高斯原语在空间和时间中的演变来生成动态场景的新视角。现有的方法通常依赖于单个多层感知器（MLP）来建模时序变形，它们往往难以一致地捕捉复杂点的运动以及精细的时间细节，尤其是在从稀疏输入视图中更是如此。我们的方法FLAG-4D通过采用一个双重形变网络解决了这个问题，该网络能够动态地将一组规范化的三维高斯模型随着时间推移转换为新的位置和各向异性形状。这个双重变形网络由即时变形网络（IDN）组成，用于建模细粒度的局部变形，并且还包含全局运动网络（GMN），用于捕捉长距离的动力学变化，通过相互学习进行优化。 为了确保这些形变既准确又在时间上平滑，FLAG-4D从一个预训练的光流主干网中整合了密集的运动特征。我们融合来自相邻时间段的运动线索，并利用一种变形引导注意机制将此流动信息与每个演变中的三维高斯模型当前状态对齐。 通过广泛的实验表明，FLAG-4D在保真度和时间上的一致性方面超越了最先进的方法，且能够更好地保留细节。
URL
https://arxiv.org/abs/2602.08558
