Abstract
Despite remarkable progress in video generation, maintaining long-term scene consistency upon revisiting previously explored areas remains challenging. Existing solutions rely either on explicitly constructing 3D geometry, which suffers from error accumulation and scale ambiguity, or on naive camera Field-of-View (FoV) retrieval, which typically fails under complex occlusions. To overcome these limitations, we propose I3DM, a novel implicit 3D-aware memory mechanism for consistent video scene generation that bypasses explicit 3D reconstruction. At the core of our approach is a 3D-aware memory retrieval strategy, which leverages the intermediate features of a pre-trained Feed-Forward Novel View Synthesis (FF-NVS) model to score view relevance, enabling robust retrieval even in highly occluded scenarios. Furthermore, to fully utilize the retrieved historical frames, we introduce a 3D-aligned memory injection module. This module implicitly warps historical content to the target view and adaptively conditions the generation on reliable warping regions, leading to improved revisit consistency and accurate camera control. Extensive experiments demonstrate that our method outperforms state-of-the-art approaches, achieving superior revisit consistency, generation fidelity, and camera control precision.
Abstract (translated)
尽管视频生成技术已取得显著进展，但在重访先前探索区域时维持长期场景一致性仍具挑战性。现有方案要么依赖显式构建3D几何结构（易产生误差累积与尺度模糊），要么采用简单的相机视场检索（在复杂遮挡场景下通常失效）。为突破这些局限，本文提出I3DM——一种用于生成一致视频场景的新型隐式3D感知记忆机制，该机制无需显式3D重建。其核心在于3D感知记忆检索策略：通过利用预训练前馈 novel view synthesis（FF-NVS）模型的中间特征对视角相关性进行评分，从而在高度遮挡场景中实现鲁棒检索。此外，为充分利用检索到的历史帧，我们设计了3D对齐记忆注入模块。该模块可将历史内容隐式扭曲至目标视角，并自适应地基于可靠扭曲区域进行生成条件控制，从而提升重访一致性与相机控制精度。大量实验表明，本方法超越现有最优方案，在重访一致性、生成保真度及相机控制精度上均取得更优表现。
URL
https://arxiv.org/abs/2603.23413
