Abstract
Generating 3D scenes from natural language holds great promise for applications in gaming, film, and design. However, existing methods struggle with automation, 3D consistency, and fine-grained control. We present DreamScene, an end-to-end framework for high-quality and editable 3D scene generation from text or dialogue. DreamScene begins with a scene planning module, where a GPT-4 agent infers object semantics and spatial constraints to construct a hybrid graph. A graph-based placement algorithm then produces a structured, collision-free layout. Based on this layout, Formation Pattern Sampling (FPS) generates object geometry using multi-timestep sampling and reconstructive optimization, enabling fast and realistic synthesis. To ensure global consistent, DreamScene employs a progressive camera sampling strategy tailored to both indoor and outdoor settings. Finally, the system supports fine-grained scene editing, including object movement, appearance changes, and 4D dynamic motion. Experiments demonstrate that DreamScene surpasses prior methods in quality, consistency, and flexibility, offering a practical solution for open-domain 3D content creation. Code and demos are available at this https URL.
Abstract (translated)
从自然语言生成三维场景在游戏、电影和设计等领域具有巨大的潜力。然而，现有的方法在自动化、三维一致性以及精细控制方面存在挑战。我们提出了DreamScene，这是一个端到端的框架，用于根据文本或对话生成高质量且可编辑的三维场景。DreamScene 以场景规划模块开始，其中GPT-4代理推断出对象语义和空间约束，并构建了一个混合图。随后，基于图的放置算法产生一个结构化且无碰撞的布局。在此布局的基础上，形成模式采样（FPS）通过多时间步采样和重构优化生成物体几何形状，从而实现快速而逼真的合成。为了确保全局一致性，DreamScene 采用了一种针对室内和室外场景量身定制的渐进式相机采样策略。最后，该系统支持精细的场景编辑功能，包括对象移动、外观变化以及四维动态运动。实验表明，DreamScene 在质量、一致性和灵活性方面超越了先前的方法，为开放领域的三维内容创作提供了一个实用解决方案。代码和演示可在[此链接](https://this-url.com)获取。
URL
https://arxiv.org/abs/2507.13985
