Abstract
3D indoor scene generation is an important problem for the design of digital and real-world environments. To automate this process, a scene generation model should be able to not only generate plausible scene layouts, but also take into consideration visual features and style preferences. Existing methods for this task exhibit very limited control over these attributes, only allowing text inputs in the form of simple object-level descriptions or pairwise spatial relationships. Our proposed method Decorum enables users to control the scene generation process with natural language by adopting language-based representations at each stage. This enables us to harness recent advancements in Large Language Models (LLMs) to model language-to-language mappings. In addition, we show that using a text-based representation allows us to select furniture for our scenes using a novel object retrieval method based on multimodal LLMs. Evaluations on the benchmark 3D-FRONT dataset show that our methods achieve improvements over existing work in text-conditioned scene synthesis and object retrieval.
Abstract (translated)
三维室内场景生成是数字和现实世界环境设计中的一个重要问题。为了实现这一过程的自动化，一个场景生成模型不仅要能够生成合理的场景布局，还应考虑视觉特征和风格偏好。目前的方法在控制这些属性方面表现出非常有限的能力，仅允许使用简单的对象级别描述或空间关系对文本输入进行操作。 我们提出的方法Decorum通过在每个阶段采用基于语言的表示形式，使用户可以用自然语言控制场景生成过程。这使得我们可以利用大型语言模型（LLMs）的最新进展来建模语言到语言的映射。此外，我们还展示了使用文本基表示法可以让我们借助一种基于多模式LLM的新颖对象检索方法为我们的场景选择家具。 在基准数据集3D-FRONT上的评估表明，与现有的以文本条件化场景合成和物体检索的方法相比，我们的方法实现了改进。
URL
https://arxiv.org/abs/2503.18155
