Abstract
In this study, we present IL3D, a large-scale dataset meticulously designed for large language model (LLM)-driven 3D scene generation, addressing the pressing demand for diverse, high-quality training data in indoor layout design. Comprising 27,816 indoor layouts across 18 prevalent room types and a library of 29,215 high-fidelity 3D object assets, IL3D is enriched with instance-level natural language annotations to support robust multimodal learning for vision-language tasks. We establish rigorous benchmarks to evaluate LLM-driven scene generation. Experimental results show that supervised fine-tuning (SFT) of LLMs on IL3D significantly improves generalization and surpasses the performance of SFT on other datasets. IL3D offers flexible multimodal data export capabilities, including point clouds, 3D bounding boxes, multiview images, depth maps, normal maps, and semantic masks, enabling seamless adaptation to various visual tasks. As a versatile and robust resource, IL3D significantly advances research in 3D scene generation and embodied intelligence, by providing high-fidelity scene data to support environment perception tasks of embodied agents.
Abstract (translated)
在这项研究中，我们介绍了IL3D，这是一个为大规模语言模型（LLM）驱动的三维场景生成精心设计的大规模数据集，旨在满足室内布局设计领域对多样化、高质量训练数据迫切需求。IL3D包含27,816个室内布局，涵盖了18种常见的房间类型，并拥有一个由29,215项高保真3D物体资产组成的库。此外，该数据集通过实例级别的自然语言注释进行了丰富，以支持视觉-语言任务中的稳健多模态学习。我们建立了严格的基准测试来评估LLM驱动的场景生成能力。实验结果显示，在IL3D上对LLMs进行监督微调（SFT）显著提高了泛化性能，并且优于在其他数据集上的SFT表现。 IL3D提供了灵活的多模态数据导出功能，包括点云、三维边界框、多视角图像、深度图、法线图和语义掩码，从而可以无缝地适应各种视觉任务。作为一个多功能且稳健的资源，IL3D通过提供高保真场景数据来支持具身智能体的环境感知任务，在三维场景生成和具身智能的研究方面取得了重大进展。
URL
https://arxiv.org/abs/2510.12095
