Abstract
We introduce SceneTransporter, an end-to-end framework for structured 3D scene generation from a single image. While existing methods generate part-level 3D objects, they often fail to organize these parts into distinct instances in open-world scenes. Through a debiased clustering probe, we reveal a critical insight: this failure stems from the lack of structural constraints within the model's internal assignment mechanism. Based on this finding, we reframe the task of structured 3D scene generation as a global correlation assignment problem. To solve this, SceneTransporter formulates and solves an entropic Optimal Transport (OT) objective within the denoising loop of the compositional DiT model. This formulation imposes two powerful structural constraints. First, the resulting transport plan gates cross-attention to enforce an exclusive, one-to-one routing of image patches to part-level 3D latents, preventing entanglement. Second, the competitive nature of the transport encourages the grouping of similar patches, a process that is further regularized by an edge-based cost, to form coherent objects and prevent fragmentation. Extensive experiments show that SceneTransporter outperforms existing methods on open-world scene generation, significantly improving instance-level coherence and geometric fidelity. Code and models will be publicly available at this https URL.
Abstract (translated)
我们介绍了一种名为SceneTransporter的端到端框架，用于从单张图像生成结构化的3D场景。尽管现有方法能够生成部分级别的3D对象，但它们往往无法在开放世界场景中将这些部分组织成不同的实例。通过偏差修正聚类探针，我们揭示了一个关键见解：这种失败的原因在于模型内部分配机制缺乏结构性约束。基于这一发现，我们将结构化3D场景的生成任务重新定义为一个全局相关性分配问题。为此，SceneTransporter在组成式DiT模型去噪循环中制定了并解决了熵最优传输（OT）目标。这种设定施加了两个强大的结构约束：首先，由此产生的运输计划通过控制跨注意力机制来确保图像补丁与部分级别3D潜在特征之间一对一的专有路由，从而防止纠缠；其次，竞争性的运输过程促使相似补丁分组，并且该过程进一步通过基于边的成本进行正则化，以形成连贯的对象并防止碎片化。广泛的实验表明，在开放世界场景生成方面，SceneTransporter超越了现有方法，显著提高了实例级别的连贯性和几何准确性。代码和模型将在[这个链接](https://this-url.com)公开提供。
URL
https://arxiv.org/abs/2602.22785
