Abstract
Compositional 3D scene generation from a single view requires the simultaneous recovery of scene layout and 3D assets. Existing approaches mainly fall into two categories: feed-forward generation methods and per-instance generation methods. The former directly predict 3D assets with explicit 6DoF poses through efficient network inference, but they generalize poorly to complex scenes. The latter improve generalization through a divide-and-conquer strategy, but suffer from time-consuming pose optimization. To bridge this gap, we introduce 3D-Fixer, a novel in-place completion paradigm. Specifically, 3D-Fixer extends 3D object generative priors to generate complete 3D assets conditioned on the partially visible point cloud at the original locations, which are cropped from the fragmented geometry obtained from the geometry estimation methods. Unlike prior works that require explicit pose alignment, 3D-Fixer uses fragmented geometry as a spatial anchor to preserve layout fidelity. At its core, we propose a coarse-to-fine generation scheme to resolve boundary ambiguity under occlusion, supported by a dual-branch conditioning network and an Occlusion-Robust Feature Alignment (ORFA) strategy for stable training. Furthermore, to address the data scarcity bottleneck, we present ARSG-110K, the largest scene-level dataset to date, comprising over 110K diverse scenes and 3M annotated images with high-fidelity 3D ground truth. Extensive experiments show that 3D-Fixer achieves state-of-the-art geometric accuracy, which significantly outperforms baselines such as MIDI and Gen3DSR, while maintaining the efficiency of the diffusion process. Code and data will be publicly available at this https URL.
Abstract (translated)
从单视图进行组合式3D场景生成需要同时恢复场景布局和3D资产。现有方法主要分为两类：前馈生成方法和逐实例生成方法。前者通过高效网络推理直接预测具有显式6自由度姿态的3D资产，但在复杂场景中泛化能力较差；后者通过分而治之策略提升泛化性，但存在姿态优化耗时的问题。为弥合这一差距，我们提出了3D-Fixer，一种新颖的就位补全范式。具体而言，3D-Fixer将3D对象生成先验扩展为在原始位置处基于部分可见点云生成完整3D资产，这些点云从几何估计方法得到的碎片化几何体中裁剪得到。与需要显式姿态对齐的先前工作不同，3D-Fixer利用碎片化几何体作为空间锚点以保持布局保真度。其核心是提出了一种由粗到细的生成方案，以解决遮挡下的边界模糊问题，并辅以双分支条件网络和遮挡鲁棒特征对齐策略以实现稳定训练。此外，为应对数据稀缺瓶颈，我们推出了迄今为止最大的场景级数据集ARSG-110K，包含超过11万个多样化场景和300万张带高保真3D真值标注的图像。大量实验表明，3D-Fixer实现了最先进的几何精度，显著优于MIDI和Gen3DSR等基线方法，同时保持了扩散过程的效率。代码和数据将在此https URL公开。
URL
https://arxiv.org/abs/2604.04406
