Abstract
Recent progress in 3D/4D scene generation emphasizes the importance of physical alignment throughout video generation and scene reconstruction. However, existing methods improve the alignment separately at each stage, making it difficult to manage subtle misalignments arising from another stage. Here, we present SteerX, a zero-shot inference-time steering method that unifies scene reconstruction into the generation process, tilting data distributions toward better geometric alignment. To this end, we introduce two geometric reward functions for 3D/4D scene generation by using pose-free feed-forward scene reconstruction models. Through extensive experiments, we demonstrate the effectiveness of SteerX in improving 3D/4D scene generation.
Abstract (translated)
最近在3D和4D场景生成方面的进展强调了在整个视频生成和场景重建过程中物理对齐的重要性。然而，现有的方法是在每个阶段单独改进对齐，这使得难以管理由其他阶段产生的微妙不对齐问题。在这里，我们介绍了SteerX，这是一种零样本推理时间引导方法，它将场景重建统一到生成过程之中，使数据分布向更好的几何对齐方向倾斜。为此，我们通过使用无姿态的前馈场景重构模型引入了两个用于3D和4D场景生成的几何奖励函数。通过广泛的实验，我们展示了SteerX在改进3D和4D场景生成方面的有效性。
URL
https://arxiv.org/abs/2503.12024
