Abstract
High-quality 3D scene generation from a single image is crucial for AR/VR and embodied AI applications. Early approaches struggle to generalize due to reliance on specialized models trained on curated small datasets. While recent advancements in large-scale 3D foundation models have significantly enhanced instance-level generation, coherent scene generation remains a challenge, where performance is limited by inaccurate per-object pose estimations and spatial inconsistency. To this end, this paper introduces CC-FMO, a zero-shot, camera-conditioned pipeline for single-image to 3D scene generation that jointly conforms to the object layout in input image and preserves instance fidelity. CC-FMO employs a hybrid instance generator that combines semantics-aware vector-set representation with detail-rich structured latent representation, yielding object geometries that are both semantically plausible and high-quality. Furthermore, CC-FMO enables the application of foundational pose estimation models in the scene generation task via a simple yet effective camera-conditioned scale-solving algorithm, to enforce scene-level coherence. Extensive experiments demonstrate that CC-FMO consistently generates high-fidelity camera-aligned compositional scenes, outperforming all state-of-the-art methods.
Abstract (translated)
从单张图像生成高质量的3D场景对于AR/VR和具身AI应用至关重要。早期的方法由于依赖于在精心策划的小型数据集上训练的专业模型，难以进行泛化。尽管近期通过大规模3D基础模型在实例级别的生成方面取得了显著进展，但连贯场景生成仍然是一个挑战，其性能受限于不准确的每个对象姿态估计和空间不一致性。为此，本文介绍了CC-FMO（相机条件下的零样本单图像到3D场景生成管道），该系统既符合输入图像中的物体布局又保持实例真实性。CC-FMO采用了一种结合语义感知向量集表示与细节丰富的结构化潜在表示的混合实例生成器，从而产生在语义上合理且高质量的对象几何形状。此外，通过简单的相机条件下的尺度解决算法，CC-FMO使基础姿态估计模型能够在场景生成任务中应用，以强制执行场景级别的连贯性。广泛的实验表明，CC-FMO能够持续地生成高保真的相机对齐组合场景，并超越所有最先进的方法。
URL
https://arxiv.org/abs/2512.00493
