Abstract
Recent advances in text-to-3D scene generation have demonstrated significant potential to transform content creation across multiple industries. Although the research community has made impressive progress in addressing the challenges of this complex task, existing methods often generate environments that are only front-facing, lack visual fidelity, exhibit limited scene understanding, and are typically fine-tuned for either indoor or outdoor settings. In this work, we address these issues and propose DreamAnywhere, a modular system for the fast generation and prototyping of 3D scenes. Our system synthesizes a 360° panoramic image from text, decomposes it into background and objects, constructs a complete 3D representation through hybrid inpainting, and lifts object masks to detailed 3D objects that are placed in the virtual environment. DreamAnywhere supports immersive navigation and intuitive object-level editing, making it ideal for scene exploration, visual mock-ups, and rapid prototyping -- all with minimal manual modeling. These features make our system particularly suitable for low-budget movie production, enabling quick iteration on scene layout and visual tone without the overhead of traditional 3D workflows. Our modular pipeline is highly customizable as it allows components to be replaced independently. Compared to current state-of-the-art text and image-based 3D scene generation approaches, DreamAnywhere shows significant improvements in coherence in novel view synthesis and achieves competitive image quality, demonstrating its effectiveness across diverse and challenging scenarios. A comprehensive user study demonstrates a clear preference for our method over existing approaches, validating both its technical robustness and practical usefulness.
Abstract (translated)
近期在文本到3D场景生成领域的进展展示了其有潜力彻底改变多个行业的内容创作。尽管研究社区在这项复杂任务上取得了令人瞩目的进步，但现有方法通常只能生成面向前方的环境，并且缺乏视觉真实感、场景理解能力有限，而且通常是为室内或室外环境进行微调的。在本工作中，我们解决了这些问题并提出了DreamAnywhere系统——这是一个用于快速生成和原型设计3D场景的模块化系统。 我们的系统从文本中合成一个360°全景图像，将其分解成背景和物体，并通过混合填充技术构建完整的3D表示。此外，它还能够将对象掩模提升为详细且放置在虚拟环境中的3D对象。DreamAnywhere支持沉浸式导航和直观的对象级编辑功能，非常适合场景探索、视觉模拟以及快速原型设计——所有这些都可以用最少的手动建模完成。 该系统的这些特点特别适合低预算电影制作，因为它允许创作者迅速迭代场景布局和视觉风格，而无需传统3D工作流程带来的额外负担。我们的模块化管道高度可定制，因为它的组件可以独立替换。与目前最先进的基于文本和图像的3D场景生成方法相比，DreamAnywhere在新视角合成的一致性和图像质量方面显示出显著改进，证明了它在各种具有挑战性的场景中的有效性。 一项全面的用户研究显示，相对于现有方法，用户更倾向于使用我们的方法，这验证了其技术稳定性和实际应用价值。
URL
https://arxiv.org/abs/2506.20367
