Abstract
Modern machine learning models for scene understanding, such as depth estimation and object tracking, rely on large, high-quality datasets that mimic real-world deployment scenarios. To address data scarcity, we propose an end-to-end system for synthetic data generation for scalable, high-quality, and customizable 3D indoor scenes. By integrating and adapting text-to-image and multi-view diffusion models with Neural Radiance Field-based meshing, this system generates highfidelity 3D object assets from text prompts and incorporates them into pre-defined floor plans using a rendering tool. By introducing novel loss functions and training strategies into existing methods, the system supports on-demand scene generation, aiming to alleviate the scarcity of current available data, generally manually crafted by artists. This system advances the role of synthetic data in addressing machine learning training limitations, enabling more robust and generalizable models for real-world applications.
Abstract (translated)
现代的机器学习模型，如场景理解中的深度估计和对象跟踪，依赖于大量高质量的数据集，这些数据集能够模拟实际部署环境。为了解决数据稀缺的问题，我们提出了一种端到端系统，用于生成可扩展、高质量且可定制的3D室内场景的合成数据。通过将文本到图像和多视角扩散模型与基于神经辐射场(meshing)的技术集成并适应，该系统可以从文本提示中生成高保真的3D对象资产，并利用渲染工具将其整合进预定义的平面图中。通过对现有方法引入新的损失函数和训练策略，此系统支持按需场景生成，旨在缓解目前由艺术家手工制作的数据稀缺问题。该系统推进了合成数据在解决机器学习训练限制中的作用，使得为实际应用开发出更稳健且更具泛化能力的模型成为可能。
URL
https://arxiv.org/abs/2504.09328
