Abstract
Recent video diffusion models demonstrate strong potential in spatial intelligence tasks due to their rich latent world priors. However, this potential is hindered by their limited controllability and geometric inconsistency, creating a gap between their strong priors and their practical use in 3D/4D tasks. As a result, current approaches often rely on retraining or fine-tuning, which risks degrading pretrained knowledge and incurs high computational costs. To address this, we propose WorldForge, a training-free, inference-time framework composed of three tightly coupled modules. Intra-Step Recursive Refinement introduces a recursive refinement mechanism during inference, which repeatedly optimizes network predictions within each denoising step to enable precise trajectory injection. Flow-Gated Latent Fusion leverages optical flow similarity to decouple motion from appearance in the latent space and selectively inject trajectory guidance into motion-related channels. Dual-Path Self-Corrective Guidance compares guided and unguided denoising paths to adaptively correct trajectory drift caused by noisy or misaligned structural signals. Together, these components inject fine-grained, trajectory-aligned guidance without training, achieving both accurate motion control and photorealistic content generation. Extensive experiments across diverse benchmarks validate our method's superiority in realism, trajectory consistency, and visual fidelity. This work introduces a novel plug-and-play paradigm for controllable video synthesis, offering a new perspective on leveraging generative priors for spatial intelligence.
Abstract (translated)
最近的视频扩散模型在空间智能任务中展示了强大的潜力，这得益于它们丰富的潜在世界先验知识。然而，这种潜力受到其控制能力有限和几何不一致性的阻碍，导致了这些强大先验与三维或四维任务实际应用之间的差距。因此，目前的方法通常依赖于重新训练或微调，这会降低预训练的知识并产生高昂的计算成本。为了解决这个问题，我们提出了WorldForge，这是一个无需训练、仅在推理时使用的框架，由三个紧密耦合的模块组成。 1. **Intra-Step Recursive Refinement**（内部递归细化）：这种机制在推断过程中引入了重复优化网络预测的功能，在每个去噪步骤内启用精确轨迹注入。 2. **Flow-Gated Latent Fusion**（基于光流的潜在融合）：利用光流相似性来解耦潜在空间中的运动和外观，并选择性地将轨迹引导注入到与运动相关的通道中。 3. **Dual-Path Self-Corrective Guidance**（双路径自校正指导）：比较指导和未指导的去噪路径，以适应由噪声或对齐不良结构信号引起的轨迹漂移。 这些组件共同作用，无需训练即可实现细粒度、轨迹对齐的引导，从而同时实现了精确运动控制和逼真内容生成。在多种基准测试中的广泛实验验证了我们的方法在现实性、轨迹一致性和视觉保真度方面的优越性。这项工作引入了一种新的即插即用范式，用于可控视频合成，并为利用生成先验知识进行空间智能提供了全新的视角。
URL
https://arxiv.org/abs/2509.15130
