Scene_Generation
-
I-Scene: 3D Instance Models are Implicit Generalizable Spatial Learners
2025-12-15 18:59:13Lu Ling, Yunhao Ge, Yichen Sheng, Aniket BeraAbstract
Generalization remains the central challenge for interactive 3D scene generation. Existing learning-based approaches ground spatial understanding in limited scene dataset, restricting generalization to new layouts. We instead reprogram a pre-trained 3D instance generator to act as a scene level learner, replacing dataset-bounded supervision with model-centric spatial supervision. This reprogramming unlocks the generator transferable spatial knowledge, enabling generalization to unseen layouts and novel object compositions. Remarkably, spatial reasoning still emerges even when the training scenes are randomly composed objects. This demonstrates that the generator's transferable scene prior provides a rich learning signal for inferring proximity, support, and symmetry from purely geometric cues. Replacing widely used canonical space, we instantiate this insight with a view-centric formulation of the scene space, yielding a fully feed-forward, generalizable scene generator that learns spatial relations directly from the instance model. Quantitative and qualitative results show that a 3D instance generator is an implicit spatial learner and reasoner, pointing toward foundation models for interactive 3D scene understanding and generation. Project page: this https URL
Abstract (translated)
泛化仍然是互动式三维场景生成的核心挑战。现有的基于学习的方法将空间理解建立在有限的场景数据集上，从而限制了其对新布局的一般化能力。我们则重新编程了一个预训练的3D实例生成器，使其作为场景级的学习者来运作，并用模型中心的空间监督取代了局限于数据集的监督。这种重编程解锁了生成器可迁移的空间知识，使它能够推广到未见过的布局和新物体组合上。值得注意的是，即使在由随机组合的对象构成的训练场景中，空间推理仍然出现。这表明生成器的可转移场景先验提供了丰富的学习信号，可以从纯粹的几何线索中推断出接近性、支撑性和对称性。 通过用以视角为中心的形式化替代广泛使用的标准空间概念，我们实现了这一洞察，并提出了一种完全前馈的、通用化的场景生成器，该生成器直接从实例模型中学得空间关系。定量和定性的结果表明，一个3D实例生成器是一个隐含的空间学习者和推理工具，这指向了用于交互式3D场景理解和生成的基础模型的发展方向。 项目页面: [此链接](this https URL)
URL
https://arxiv.org/abs/2512.13683
PDF
-
RoomPilot: Controllable Synthesis of Interactive Indoor Environments via Multimodal Semantic Parsing
2025-12-12 02:33:09Wentang Chen, Shougao Zhang, Yiman Zhang, Tianhao Zhou, Ruihui LiAbstract
Generating controllable and interactive indoor scenes is fundamental to applications in game development, architectural visualization, and embodied AI training. Yet existing approaches either handle a narrow range of input modalities or rely on stochastic processes that hinder controllability. To overcome these limitations, we introduce RoomPilot, a unified framework that parses diverse multi-modal inputs--textual descriptions or CAD floor plans--into an Indoor Domain-Specific Language (IDSL) for indoor structured scene generation. The key insight is that a well-designed IDSL can act as a shared semantic representation, enabling coherent, high-quality scene synthesis from any single modality while maintaining interaction semantics. In contrast to conventional procedural methods that produce visually plausible but functionally inert layouts, RoomPilot leverages a curated dataset of interaction-annotated assets to synthesize environments exhibiting realistic object behaviors. Extensive experiments further validate its strong multi-modal understanding, fine-grained controllability in scene generation, and superior physical consistency and visual fidelity, marking a significant step toward general-purpose controllable 3D indoor scene generation.
Abstract (translated)
生成可控且互动的室内场景对于游戏开发、建筑可视化以及具身人工智能训练等应用来说至关重要。然而，现有的方法要么只能处理有限种类的输入模态（如文本描述或CAD平面图），要么依赖于随机过程来创建这些场景，这使得难以实现精确控制。为了克服这些问题，我们引入了RoomPilot——一个统一框架，它能够解析多种多样的多模态输入（包括文本描述和CAD楼层计划）并将其转换为室内专用领域的语言（IDSL），进而生成结构化的室内场景。关键的理念是：经过精心设计的IDSL可以作为共享语义表示，从单一模态中生成一致且高质量的场景，并保持交互性。 与传统的程序化方法不同，这些方法虽然能够产生视觉上可信但功能相对静止的布局，RoomPilot利用一个注有互动数据的数据集来合成表现出真实物体行为的环境。大量的实验进一步验证了它在多模态理解、场景生成中的精细控制以及物理一致性和视觉逼真度方面的卓越表现，标志着向着通用可控3D室内场景生成迈出了一大步。
URL
https://arxiv.org/abs/2512.11234
PDF
-
SceneMaker: Open-set 3D Scene Generation with Decoupled De-occlusion and Pose Estimation Model
2025-12-11 18:59:56Yukai Shi, Weiyu Li, Zihao Wang, Hongyang Li, Xingyu Chen, Ping Tan, Lei ZhangAbstract
We propose a decoupled 3D scene generation framework called SceneMaker in this work. Due to the lack of sufficient open-set de-occlusion and pose estimation priors, existing methods struggle to simultaneously produce high-quality geometry and accurate poses under severe occlusion and open-set settings. To address these issues, we first decouple the de-occlusion model from 3D object generation, and enhance it by leveraging image datasets and collected de-occlusion datasets for much more diverse open-set occlusion patterns. Then, we propose a unified pose estimation model that integrates global and local mechanisms for both self-attention and cross-attention to improve accuracy. Besides, we construct an open-set 3D scene dataset to further extend the generalization of the pose estimation model. Comprehensive experiments demonstrate the superiority of our decoupled framework on both indoor and open-set scenes. Our codes and datasets is released at this https URL.
Abstract (translated)
在本文中，我们提出了一种称为SceneMaker的解耦3D场景生成框架。由于缺乏足够的开放集去遮挡和姿态估计先验知识，在严重遮挡和开放集设置下同时产生高质量几何形状和准确的姿态方面，现有方法面临挑战。为了解决这些问题，我们首先将去遮挡模型从三维物体生成中分离出来，并通过利用图像数据集和收集的去遮挡数据集来增强其多样性，以处理更加多样化的开放集遮挡模式。此外，我们提出了一种统一的姿态估计模型，该模型结合了全局和局部机制的自我注意和交叉注意力，以提高准确性。另外，我们构建了一个开放集3D场景数据集，进一步扩展姿态估计模型的泛化能力。全面的实验表明，在室内和开放集场景中，我们的解耦框架具有优越性。我们的代码和数据集可在[此处](https://此URL/)获取。
URL
https://arxiv.org/abs/2512.10957
PDF
-
Self-Evolving 3D Scene Generation from a Single Image
2025-12-09 18:44:21Kaizhi Zheng, Yue Fan, Jing Gu, Zishuo Xu, Xuehai He, Xin Eric WangAbstract
Generating high-quality, textured 3D scenes from a single image remains a fundamental challenge in vision and graphics. Recent image-to-3D generators recover reasonable geometry from single views, but their object-centric training limits generalization to complex, large-scale scenes with faithful structure and texture. We present EvoScene, a self-evolving, training-free framework that progressively reconstructs complete 3D scenes from single images. The key idea is combining the complementary strengths of existing models: geometric reasoning from 3D generation models and visual knowledge from video generation models. Through three iterative stages--Spatial Prior Initialization, Visual-guided 3D Scene Mesh Generation, and Spatial-guided Novel View Generation--EvoScene alternates between 2D and 3D domains, gradually improving both structure and appearance. Experiments on diverse scenes demonstrate that EvoScene achieves superior geometric stability, view-consistent textures, and unseen-region completion compared to strong baselines, producing ready-to-use 3D meshes for practical applications.
Abstract (translated)
从单张图像生成高质量、具有纹理的三维场景仍然是计算机视觉和图形学中的一个基本挑战。最近的一些图像到三维模型的生成器能够从单一视角恢复合理的几何结构，但它们以对象为中心的训练方式限制了其在复杂大规模场景中实现忠实结构和纹理的一般化能力。我们提出了EvoScene，这是一个自我演化的、无需训练的框架，它可以逐步地从单张图片中重建完整的三维场景。核心思想是结合现有模型的优势：即三维生成模型中的几何推理能力和视频生成模型中的视觉知识。 通过三个迭代阶段——空间先验初始化、基于视觉引导的三维场景网格生成和基于空间引导的新视角生成——EvoScene在二维和三维域之间交替进行，逐步提升结构和外观的质量。实验结果表明，在多样化的场景中，EvoScene相比于强大的基准模型实现了更优的几何稳定性、视图一致性的纹理以及未见区域的完成度，并且可以生成适合实际应用的即用型3D网格。
URL
https://arxiv.org/abs/2512.08905
PDF
-
Optimization-Guided Diffusion for Interactive Scene Generation
2025-12-08 15:56:18Shiaho Li, Naisheng Ye, Tianyu Li, Kashyap Chitta, Tuo An, Peng Su, Boyang Wang, Haiou Liu, Chen Lv, Hongyang LiAbstract
Realistic and diverse multi-agent driving scenes are crucial for evaluating autonomous vehicles, but safety-critical events which are essential for this task are rare and underrepresented in driving datasets. Data-driven scene generation offers a low-cost alternative by synthesizing complex traffic behaviors from existing driving logs. However, existing models often lack controllability or yield samples that violate physical or social constraints, limiting their usability. We present OMEGA, an optimization-guided, training-free framework that enforces structural consistency and interaction awareness during diffusion-based sampling from a scene generation model. OMEGA re-anchors each reverse diffusion step via constrained optimization, steering the generation towards physically plausible and behaviorally coherent trajectories. Building on this framework, we formulate ego-attacker interactions as a game-theoretic optimization in the distribution space, approximating Nash equilibria to generate realistic, safety-critical adversarial scenarios. Experiments on nuPlan and Waymo show that OMEGA improves generation realism, consistency, and controllability, increasing the ratio of physically and behaviorally valid scenes from 32.35% to 72.27% for free exploration capabilities, and from 11% to 80% for controllability-focused generation. Our approach can also generate $5\times$ more near-collision frames with a time-to-collision under three seconds while maintaining the overall scene realism.
Abstract (translated)
真实且多样化的多代理驾驶场景对于评估自动驾驶车辆至关重要，但这些场景中关键的安全事件（对任务来说是必不可少的）在驾驶数据集中非常罕见并且代表性不足。基于数据驱动的场景生成提供了一种低成本的选择，通过从现有的驾驶日志中合成复杂的交通行为来实现这一点。然而，现有模型往往缺乏可控性或产生违反物理和社会约束的样本，限制了它们的应用。 我们提出了OMEGA框架，这是一个优化引导、无需训练的框架，在基于扩散的方法进行场景生成采样过程中强制执行结构一致性和交互意识。通过在每个反向扩散步骤中应用有约束条件的优化方法，OMEGA重新定位每次迭代，使生成的过程朝着物理上合理且行为连贯的方向前进。 在此框架基础上，我们将自我车辆与攻击者之间的互动作为分布空间中的博弈论优化问题来处理，逼近纳什均衡以生成现实且具有安全挑战性的对抗性场景。在nuPlan和Waymo数据集上的实验表明，OMEGA提高了生成的场景的真实感、一致性和可控性，在自由探索能力方面将物理和行为上有效场景的比例从32.35%提升至72.27%，而在聚焦于可控性生成的情况下，这一比例则从11%大幅提升到80%。此外，我们的方法还能在保持整体场景真实性的前提下，产生五倍数量的接近碰撞帧，这些帧的时间距离（time-to-collision）小于三秒。 通过这样的改进，OMEGA不仅提高了合成驾驶场景的质量和真实性，还为自动驾驶车辆的安全性和性能评估提供了更有价值的数据。
URL
https://arxiv.org/abs/2512.07661
PDF
-
MoCA: Mixture-of-Components Attention for Scalable Compositional 3D Generation
2025-12-08 15:17:01Zhiqi Li, Wenhuan Li, Tengfei Wang, Zhenwei Wang, Junta Wu, Haoyuan Wang, Yunhan Yang, Zehuan Huang, Yang Li, Peidong Liu, Chunchao GuoAbstract
Compositionality is critical for 3D object and scene generation, but existing part-aware 3D generation methods suffer from poor scalability due to quadratic global attention costs when increasing the number of components. In this work, we present MoCA, a compositional 3D generative model with two key designs: (1) importance-based component routing that selects top-k relevant components for sparse global attention, and (2) unimportant components compression that preserve contextual priors of unselected components while reducing computational complexity of global attention. With these designs, MoCA enables efficient, fine-grained compositional 3D asset creation with scalable number of components. Extensive experiments show MoCA outperforms baselines on both compositional object and scene generation tasks. Project page: this https URL
Abstract (translated)
组合性对于生成三维物体和场景至关重要，但现有的具有部件感知能力的3D生成方法在增加组件数量时由于二次全局注意力成本而面临扩展性的挑战。在这项工作中，我们提出了MoCA，这是一种组合式3D生成模型，具备两个关键设计：(1) 基于重要性进行组件路由，选择前k个相关的组件用于稀疏的全局注意力；(2) 对不重要的组件进行压缩，在减少全局注意力计算复杂度的同时保留未选组件的上下文先验。通过这两个设计，MoCA能够实现高效、细粒度的组合式3D资产创建，并且支持可扩展数量的组件。大量的实验表明，MoCA在组合式物体和场景生成任务上超越了基准方法。项目页面：[此链接](this https URL)
URL
https://arxiv.org/abs/2512.07628
PDF
-
MVRoom: Controllable 3D Indoor Scene Generation with Multi-View Diffusion Models
2025-12-03 20:33:18Shaoheng Fang, Chaohui Yu, Fan Wang, Qixing HuangAbstract
We introduce MVRoom, a controllable novel view synthesis (NVS) pipeline for 3D indoor scenes that uses multi-view diffusion conditioned on a coarse 3D layout. MVRoom employs a two-stage design in which the 3D layout is used throughout to enforce multi-view consistency. The first stage employs novel representations to effectively bridge the 3D layout and consistent image-based condition signals for multi-view generation. The second stage performs image-conditioned multi-view generation, incorporating a layout-aware epipolar attention mechanism to enhance multi-view consistency during the diffusion process. Additionally, we introduce an iterative framework that generates 3D scenes with varying numbers of objects and scene complexities by recursively performing multi-view generation (MVRoom), supporting text-to-scene generation. Experimental results demonstrate that our approach achieves high-fidelity and controllable 3D scene generation for NVS, outperforming state-of-the-art baseline methods both quantitatively and qualitatively. Ablation studies further validate the effectiveness of key components within our generation pipeline.
Abstract (translated)
我们介绍了MVRoom，这是一种用于室内三维场景的可控新颖视角合成（NVS）流水线，它使用基于粗略3D布局的多视图扩散。MVRoom采用两阶段设计，在整个过程中利用3D布局来强制执行多视图一致性。第一阶段采用了新的表示方法，有效地将3D布局与一致的图像条件信号结合起来，用于多视图生成。第二阶段进行受图像条件限制的多视图生成，并引入了一种具有视角意识的极线注意力机制，在扩散过程中增强多视图的一致性。 此外，我们还介绍了一个迭代框架，该框架通过递归执行多视图生成（MVRoom）来生成包含不同数量对象和场景复杂度的3D场景，支持从文本到场景的生成。实验结果表明，我们的方法在NVS方面实现了高质量且可控的三维场景生成，在定量和定性评估中均优于现有的基准方法。消融研究进一步验证了我们生成管道中关键组件的有效性。
URL
https://arxiv.org/abs/2512.04248
PDF
-
TabletopGen: Instance-Level Interactive 3D Tabletop Scene Generation from Text or Single Image
2025-12-01 02:38:52Ziqian Wang, Yonghao He, Licheng Yang, Wei Zou, Hongxuan Ma, Liu Liu, Wei Sui, Yuxin Guo, Hu SuAbstract
Generating high-fidelity, physically interactive 3D simulated tabletop scenes is essential for embodied AI--especially for robotic manipulation policy learning and data synthesis. However, current text- or image-driven 3D scene generation methods mainly focus on large-scale scenes, struggling to capture the high-density layouts and complex spatial relations that characterize tabletop scenes. To address these challenges, we propose TabletopGen, a training-free, fully automatic framework that generates diverse, instance-level interactive 3D tabletop scenes. TabletopGen accepts a reference image as input, which can be synthesized by a text-to-image model to enhance scene diversity. We then perform instance segmentation and completion on the reference to obtain per-instance images. Each instance is reconstructed into a 3D model followed by canonical coordinate alignment. The aligned 3D models then undergo pose and scale estimation before being assembled into a collision-free, simulation-ready tabletop scene. A key component of our framework is a novel pose and scale alignment approach that decouples the complex spatial reasoning into two stages: a Differentiable Rotation Optimizer for precise rotation recovery and a Top-view Spatial Alignment mechanism for robust translation and scale estimation, enabling accurate 3D reconstruction from 2D reference. Extensive experiments and user studies show that TabletopGen achieves state-of-the-art performance, markedly surpassing existing methods in visual fidelity, layout accuracy, and physical plausibility, capable of generating realistic tabletop scenes with rich stylistic and spatial diversity. Our code will be publicly available.
Abstract (translated)
生成高保真、物理互动的3D桌面仿真场景对于具身人工智能（embodied AI）至关重要，尤其是在机器人操作策略学习和数据合成方面。然而，当前基于文本或图像驱动的3D场景生成方法主要关注大规模场景的创建，难以捕捉到桌面场景中高密度布局和复杂空间关系的特点。为了应对这些挑战，我们提出了TabletopGen，这是一个无需训练、完全自动化的框架，可以生成多样化的实例级别的互动3D桌面场景。TabletopGen接受一张参考图像作为输入，该图像是通过文本到图像的模型合成而来，以增强场景多样性。然后，在参考图像上进行实例分割和补全操作，获取每个实例对应的图片。接着将每一个实例重构为一个3D模型，并进行标准坐标对齐。对齐后的3D模型会经历姿态与尺度估计过程，最后组装成一个无碰撞的、可以模拟的真实桌面场景。 我们框架的一个关键组成部分是新的姿态和尺度对齐方法，它将复杂的空间推理分解为两个阶段：第一阶段使用可微旋转优化器进行精确旋转恢复；第二阶段则采用俯视图空间对齐机制来稳健地估计位移与缩放比例。这种方法使得从2D参考图像到3D重建的准确度得到提升。 通过广泛的实验和用户研究，TabletopGen在视觉保真度、布局准确性以及物理真实性方面达到了最先进的水平，显著超越了现有方法，并能够生成具有丰富风格和空间多样性的现实桌面场景。我们的代码将公开发布。
URL
https://arxiv.org/abs/2512.01204
PDF
-
CC-FMO: Camera-Conditioned Zero-Shot Single Image to 3D Scene Generation with Foundation Model Orchestration
2025-11-29 14:01:13Boshi Tang, Henry Zheng, Rui Huang, Gao HuangAbstract
High-quality 3D scene generation from a single image is crucial for AR/VR and embodied AI applications. Early approaches struggle to generalize due to reliance on specialized models trained on curated small datasets. While recent advancements in large-scale 3D foundation models have significantly enhanced instance-level generation, coherent scene generation remains a challenge, where performance is limited by inaccurate per-object pose estimations and spatial inconsistency. To this end, this paper introduces CC-FMO, a zero-shot, camera-conditioned pipeline for single-image to 3D scene generation that jointly conforms to the object layout in input image and preserves instance fidelity. CC-FMO employs a hybrid instance generator that combines semantics-aware vector-set representation with detail-rich structured latent representation, yielding object geometries that are both semantically plausible and high-quality. Furthermore, CC-FMO enables the application of foundational pose estimation models in the scene generation task via a simple yet effective camera-conditioned scale-solving algorithm, to enforce scene-level coherence. Extensive experiments demonstrate that CC-FMO consistently generates high-fidelity camera-aligned compositional scenes, outperforming all state-of-the-art methods.
Abstract (translated)
从单张图像生成高质量的3D场景对于AR/VR和具身AI应用至关重要。早期的方法由于依赖于在精心策划的小型数据集上训练的专业模型，难以进行泛化。尽管近期通过大规模3D基础模型在实例级别的生成方面取得了显著进展，但连贯场景生成仍然是一个挑战，其性能受限于不准确的每个对象姿态估计和空间不一致性。为此，本文介绍了CC-FMO（相机条件下的零样本单图像到3D场景生成管道），该系统既符合输入图像中的物体布局又保持实例真实性。CC-FMO采用了一种结合语义感知向量集表示与细节丰富的结构化潜在表示的混合实例生成器，从而产生在语义上合理且高质量的对象几何形状。此外，通过简单的相机条件下的尺度解决算法，CC-FMO使基础姿态估计模型能够在场景生成任务中应用，以强制执行场景级别的连贯性。广泛的实验表明，CC-FMO能够持续地生成高保真的相机对齐组合场景，并超越所有最先进的方法。
URL
https://arxiv.org/abs/2512.00493
PDF
-
GeoWorld: Unlocking the Potential of Geometry Models to Facilitate High-Fidelity 3D Scene Generation
2025-11-28 13:55:45Yuhao Wan, Lijuan Liu, Jingzhi Zhou, Zihan Zhou, Xuying Zhang, Dongbo Zhang, Shaohui Jiao, Qibin Hou, Ming-Ming ChengAbstract
Previous works leveraging video models for image-to-3D scene generation tend to suffer from geometric distortions and blurry content. In this paper, we renovate the pipeline of image-to-3D scene generation by unlocking the potential of geometry models and present our GeoWorld. Instead of exploiting geometric information obtained from a single-frame input, we propose to first generate consecutive video frames and then take advantage of the geometry model to provide full-frame geometry features, which contain richer information than single-frame depth maps or camera embeddings used in previous methods, and use these geometry features as geometrical conditions to aid the video generation model. To enhance the consistency of geometric structures, we further propose a geometry alignment loss to provide the model with real-world geometric constraints and a geometry adaptation module to ensure the effective utilization of geometry features. Extensive experiments show that our GeoWorld can generate high-fidelity 3D scenes from a single image and a given camera trajectory, outperforming prior methods both qualitatively and quantitatively. Project Page: this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2511.23191
PDF
-
PAT3D: Physics-Augmented Text-to-3D Scene Generation
2025-11-26 23:23:58Guying Lin, Kemeng Huang, Michael Liu, Ruihan Gao, Hanke Chen, Lyuhao Chen, Beijia Lu, Taku Komura, Yuan Liu, Jun-Yan Zhu, Minchen LiAbstract
We introduce PAT3D, the first physics-augmented text-to-3D scene generation framework that integrates vision-language models with physics-based simulation to produce physically plausible, simulation-ready, and intersection-free 3D scenes. Given a text prompt, PAT3D generates 3D objects, infers their spatial relations, and organizes them into a hierarchical scene tree, which is then converted into initial conditions for simulation. A differentiable rigid-body simulator ensures realistic object interactions under gravity, driving the scene toward static equilibrium without interpenetrations. To further enhance scene quality, we introduce a simulation-in-the-loop optimization procedure that guarantees physical stability and non-intersection, while improving semantic consistency with the input prompt. Experiments demonstrate that PAT3D substantially outperforms prior approaches in physical plausibility, semantic consistency, and visual quality. Beyond high-quality generation, PAT3D uniquely enables simulation-ready 3D scenes for downstream tasks such as scene editing and robotic manipulation. Code and data will be released upon acceptance.
Abstract (translated)
URL
https://arxiv.org/abs/2511.21978
PDF
-
LaGen: Towards Autoregressive LiDAR Scene Generation
2025-11-26 10:39:16Sizhuo Zhou, Xiaosong Jia, Fanrui Zhang, Junjie Li, Juyong Zhang, Yukang Feng, Jianwen Sun, Songbur Wong, Junqi You, Junchi YanAbstract
Generative world models for autonomous driving (AD) have become a trending topic. Unlike the widely studied image modality, in this work we explore generative world models for LiDAR data. Existing generation methods for LiDAR data only support single frame generation, while existing prediction approaches require multiple frames of historical input and can only deterministically predict multiple frames at once, lacking interactivity. Both paradigms fail to support long-horizon interactive generation. To this end, we introduce LaGen, which to the best of our knowledge is the first framework capable of frame-by-frame autoregressive generation of long-horizon LiDAR scenes. LaGen is able to take a single-frame LiDAR input as a starting point and effectively utilize bounding box information as conditions to generate high-fidelity 4D scene point clouds. In addition, we introduce a scene decoupling estimation module to enhance the model's interactive generation capability for object-level content, as well as a noise modulation module to mitigate error accumulation during long-horizon generation. We construct a protocol based on nuScenes for evaluating long-horizon LiDAR scene generation. Experimental results comprehensively demonstrate LaGen outperforms state-of-the-art LiDAR generation and prediction models, especially on the later frames.
Abstract (translated)
URL
https://arxiv.org/abs/2511.21256
PDF
-
MarketGen: A Scalable Simulation Platform with Auto-Generated Embodied Supermarket Environments
2025-11-26 08:20:20Xu Hu, Yiyang Feng, Junran Peng, Jiawei He, Liyi Chen, Chuanchen Luo, Xucheng Yin, Qing Li, Zhaoxiang ZhangAbstract
The development of embodied agents for complex commercial environments is hindered by a critical gap in existing robotics datasets and benchmarks, which primarily focus on household or tabletop settings with short-horizon tasks. To address this limitation, we introduce MarketGen, a scalable simulation platform with automatic scene generation for complex supermarket environments. MarketGen features a novel agent-based Procedural Content Generation (PCG) framework. It uniquely supports multi-modal inputs (text and reference images) and integrates real-world design principles to automatically generate complete, structured, and realistic supermarkets. We also provide an extensive and diverse 3D asset library with a total of 1100+ supermarket goods and parameterized facilities assets. Building on this generative foundation, we propose a novel benchmark for assessing supermarket agents, featuring two daily tasks in a supermarket: (1) Checkout Unloading: long-horizon tabletop tasks for cashier agents, and (2) In-Aisle Item Collection: complex mobile manipulation tasks for salesperson agents. We validate our platform and benchmark through extensive experiments, including the deployment of a modular agent system and successful sim-to-real transfer. MarketGen provides a comprehensive framework to accelerate research in embodied AI for complex commercial applications.
Abstract (translated)
URL
https://arxiv.org/abs/2511.21161
PDF
-
FaithFusion: Harmonizing Reconstruction and Generation via Pixel-wise Information Gain
2025-11-26 06:58:57YuAn Wang, Xiaofan Li, Chi Huang, Wenhao Zhang, Hao Li, Bosheng Wang, Xun Sun, Jun WangAbstract
In controllable driving-scene reconstruction and 3D scene generation, maintaining geometric fidelity while synthesizing visually plausible appearance under large viewpoint shifts is crucial. However, effective fusion of geometry-based 3DGS and appearance-driven diffusion models faces inherent challenges, as the absence of pixel-wise, 3D-consistent editing criteria often leads to over-restoration and geometric drift. To address these issues, we introduce \textbf{FaithFusion}, a 3DGS-diffusion fusion framework driven by pixel-wise Expected Information Gain (EIG). EIG acts as a unified policy for coherent spatio-temporal synthesis: it guides diffusion as a spatial prior to refine high-uncertainty regions, while its pixel-level weighting distills the edits back into 3DGS. The resulting plug-and-play system is free from extra prior conditions and structural this http URL experiments on the Waymo dataset demonstrate that our approach attains SOTA performance across NTA-IoU, NTL-IoU, and FID, maintaining an FID of 107.47 even at 6 meters lane shift. Our code is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2511.21113
PDF
-
FilmSceneDesigner: Chaining Set Design for Procedural Film Scene Generation
2025-11-24 14:00:40Zhifeng Xie, Keyi Zhang, Yiye Yan, Yuling Guo, Fan Yang, Jiting Zhou, Mengtian LiAbstract
Film set design plays a pivotal role in cinematic storytelling and shaping the visual atmosphere. However, the traditional process depends on expert-driven manual modeling, which is labor-intensive and time-consuming. To address this issue, we introduce FilmSceneDesigner, an automated scene generation system that emulates professional film set design workflow. Given a natural language description, including scene type, historical period, and style, we design an agent-based chaining framework to generate structured parameters aligned with film set design workflow, guided by prompt strategies that ensure parameter accuracy and coherence. On the other hand, we propose a procedural generation pipeline which executes a series of dedicated functions with the structured parameters for floorplan and structure generation, material assignment, door and window placement, and object retrieval and layout, ultimately constructing a complete film scene from scratch. Moreover, to enhance cinematic realism and asset diversity, we construct SetDepot-Pro, a curated dataset of 6,862 film-specific 3D assets and 733 materials. Experimental results and human evaluations demonstrate that our system produces structurally sound scenes with strong cinematic fidelity, supporting downstream tasks such as virtual previs, construction drawing and mood board creation.
Abstract (translated)
URL
https://arxiv.org/abs/2511.19137
PDF
-
A Self-Conditioned Representation Guided Diffusion Model for Realistic Text-to-LiDAR Scene Generation
2025-11-24 11:32:15Wentao Qu, Guofeng Mei, Yang Wu, Yongshun Gong, Xiaoshui Huang, Liang XiaoAbstract
Text-to-LiDAR generation can customize 3D data with rich structures and diverse scenes for downstream tasks. However, the scarcity of Text-LiDAR pairs often causes insufficient training priors, generating overly smooth 3D scenes. Moreover, low-quality text descriptions may degrade generation quality and controllability. In this paper, we propose a Text-to-LiDAR Diffusion Model for scene generation, named T2LDM, with a Self-Conditioned Representation Guidance (SCRG). Specifically, SCRG, by aligning to the real representations, provides the soft supervision with reconstruction details for the Denoising Network (DN) in training, while decoupled in inference. In this way, T2LDM can perceive rich geometric structures from data distribution, generating detailed objects in scenes. Meanwhile, we construct a content-composable Text-LiDAR benchmark, T2nuScenes, along with a controllability metric. Based on this, we analyze the effects of different text prompts for LiDAR generation quality and controllability, providing practical prompt paradigms and insights. Furthermore, a directional position prior is designed to mitigate street distortion, further improving scene fidelity. Additionally, by learning a conditional encoder via frozen DN, T2LDM can support multiple conditional tasks, including Sparse-to-Dense, Dense-to-Sparse, and Semantic-to-LiDAR generation. Extensive experiments in unconditional and conditional generation demonstrate that T2LDM outperforms existing methods, achieving state-of-the-art scene generation.
Abstract (translated)
URL
https://arxiv.org/abs/2511.19004
PDF
-
Yo'City: Personalized and Boundless 3D Realistic City Scene Generation via Self-Critic Expansion
2025-11-24 04:02:48Keyang Lu, Sifan Zhou, Hongbin Xu, Gang Xu, Zhifei Yang, Yikai Wang, Zhen Xiao, Jieyi Long, Ming LiAbstract
Realistic 3D city generation is fundamental to a wide range of applications, including virtual reality and digital twins. However, most existing methods rely on training a single diffusion model, which limits their ability to generate personalized and boundless city-scale scenes. In this paper, we present Yo'City, a novel agentic framework that enables user-customized and infinitely expandable 3D city generation by leveraging the reasoning and compositional capabilities of off-the-shelf large models. Specifically, Yo'City first conceptualize the city through a top-down planning strategy that defines a hierarchical "City-District-Grid" structure. The Global Planner determines the overall layout and potential functional districts, while the Local Designer further refines each district with detailed grid-level descriptions. Subsequently, the grid-level 3D generation is achieved through a "produce-refine-evaluate" isometric image synthesis loop, followed by image-to-3D generation. To simulate continuous city evolution, Yo'City further introduces a user-interactive, relationship-guided expansion mechanism, which performs scene graph-based distance- and semantics-aware layout optimization, ensuring spatially coherent city growth. To comprehensively evaluate our method, we construct a diverse benchmark dataset and design six multi-dimensional metrics that assess generation quality from the perspectives of semantics, geometry, texture, and layout. Extensive experiments demonstrate that Yo'City consistently outperforms existing state-of-the-art methods across all evaluation aspects.
Abstract (translated)
URL
https://arxiv.org/abs/2511.18734
PDF
-
GEN3D: Generating Domain-Free 3D Scenes from a Single Image
2025-11-18 09:40:43Yuxin Zhang, Ziyu Lu, Hongbo Duan, Keyu Fan, Pengting Luo, Peiyu Zhuang, Mengyu Yang, Houde LiuAbstract
Despite recent advancements in neural 3D reconstruction, the dependence on dense multi-view captures restricts their broader applicability. Additionally, 3D scene generation is vital for advancing embodied AI and world models, which depend on diverse, high-quality scenes for learning and evaluation. In this work, we propose Gen3d, a novel method for generation of high-quality, wide-scope, and generic 3D scenes from a single image. After the initial point cloud is created by lifting the RGBD image, Gen3d maintains and expands its world model. The 3D scene is finalized through optimizing a Gaussian splatting representation. Extensive experiments on diverse datasets demonstrate the strong generalization capability and superior performance of our method in generating a world model and Synthesizing high-fidelity and consistent novel views.
Abstract (translated)
URL
https://arxiv.org/abs/2511.14291
PDF
-
DriveLiDAR4D: Sequential and Controllable LiDAR Scene Generation for Autonomous Driving
2025-11-17 12:43:26Kaiwen Cai, Xinze Liu, Xia Zhou, Hengtong Hu, Jie Xiang, Luyao Zhang, Xueyang Zhang, Kun Zhan, Yifei Zhan, Xianpeng LangAbstract
The generation of realistic LiDAR point clouds plays a crucial role in the development and evaluation of autonomous driving systems. Although recent methods for 3D LiDAR point cloud generation have shown significant improvements, they still face notable limitations, including the lack of sequential generation capabilities and the inability to produce accurately positioned foreground objects and realistic backgrounds. These shortcomings hinder their practical applicability. In this paper, we introduce DriveLiDAR4D, a novel LiDAR generation pipeline consisting of multimodal conditions and a novel sequential noise prediction model LiDAR4DNet, capable of producing temporally consistent LiDAR scenes with highly controllable foreground objects and realistic backgrounds. To the best of our knowledge, this is the first work to address the sequential generation of LiDAR scenes with full scene manipulation capability in an end-to-end manner. We evaluated DriveLiDAR4D on the nuScenes and KITTI datasets, where we achieved an FRD score of 743.13 and an FVD score of 16.96 on the nuScenes dataset, surpassing the current state-of-the-art (SOTA) method, UniScene, with an performance boost of 37.2% in FRD and 24.1% in FVD, respectively.
Abstract (translated)
URL
https://arxiv.org/abs/2511.13309
PDF
-
Omni-View: Unlocking How Generation Facilitates Understanding in Unified 3D Model based on Multiview images
2025-11-10 15:44:48JiaKui Hu, Shanshan Zhao, Qing-Guo Chen, Xuerui Qiu, Jialun Liu, Zhao Xu, Weihua Luo, Kaifu Zhang, Yanye LuAbstract
This paper presents Omni-View, which extends the unified multimodal understanding and generation to 3D scenes based on multiview images, exploring the principle that "generation facilitates understanding". Consisting of understanding model, texture module, and geometry module, Omni-View jointly models scene understanding, novel view synthesis, and geometry estimation, enabling synergistic interaction between 3D scene understanding and generation tasks. By design, it leverages the spatiotemporal modeling capabilities of its texture module responsible for appearance synthesis, alongside the explicit geometric constraints provided by its dedicated geometry module, thereby enriching the model's holistic understanding of 3D scenes. Trained with a two-stage strategy, Omni-View achieves a state-of-the-art score of 55.4 on the VSI-Bench benchmark, outperforming existing specialized 3D understanding models, while simultaneously delivering strong performance in both novel view synthesis and 3D scene generation.
Abstract (translated)
URL
https://arxiv.org/abs/2511.07222
PDF
