Abstract
Real-world scenes, such as those in ScanNet, are difficult to capture, with highly limited data available. Generating realistic scenes with varied object poses remains an open and challenging task. In this work, we propose FactoredScenes, a framework that synthesizes realistic 3D scenes by leveraging the underlying structure of rooms while learning the variation of object poses from lived-in scenes. We introduce a factored representation that decomposes scenes into hierarchically organized concepts of room programs and object poses. To encode structure, FactoredScenes learns a library of functions capturing reusable layout patterns from which scenes are drawn, then uses large language models to generate high-level programs, regularized by the learned library. To represent scene variations, FactoredScenes learns a program-conditioned model to hierarchically predict object poses, and retrieves and places 3D objects in a scene. We show that FactoredScenes generates realistic, real-world rooms that are difficult to distinguish from real ScanNet scenes.
Abstract (translated)
现实世界的场景，如ScanNet中的场景，难以捕捉，且可用的数据非常有限。生成具有多变物体姿态的逼真场景仍然是一个开放且具挑战性的任务。在这项工作中，我们提出了FactoredScenes框架，该框架通过利用房间的基础结构，并从日常生活场景中学习物体姿势的变化来合成逼真的3D场景。我们引入了一种分层的概念分解表示方法，将场景拆分为房间程序和物体姿态的组织化概念。 为了编码结构，FactoredScenes 学习了一个函数库，捕捉可重复使用的布局模式，然后利用大型语言模型生成高级程序，并通过学习到的库进行正则化处理。为了表示场景的变化，FactoredScenes 学习了一个条件于程序的模型来分层次预测物体姿态，并在场景中检索和放置3D对象。 我们展示了FactoredScenes能够生成难以与真实的ScanNet房间相区分的真实世界房间，证明了其有效性和创新性。
URL
https://arxiv.org/abs/2510.10292
