Abstract
Procedural generation techniques in 3D rendering engines have revolutionized the creation of complex environments, reducing reliance on manual design. Recent approaches using Large Language Models (LLMs) for 3D scene generation show promise but often lack domain-specific reasoning, verification mechanisms, and modular design. These limitations lead to reduced control and poor scalability. This paper investigates the use of LLMs to generate agricultural synthetic simulation environments from natural language prompts, specifically to address the limitations of lacking domain-specific reasoning, verification mechanisms, and modular design. A modular multi-LLM pipeline was developed, integrating 3D asset retrieval, domain knowledge injection, and code generation for the Unreal rendering engine using its API. This results in a 3D environment with realistic planting layouts and environmental context, all based on the input prompt and the domain knowledge. To enhance accuracy and scalability, the system employs a hybrid strategy combining LLM optimization techniques such as few-shot prompting, Retrieval-Augmented Generation (RAG), finetuning, and validation. Unlike monolithic models, the modular architecture enables structured data handling, intermediate verification, and flexible expansion. The system was evaluated using structured prompts and semantic accuracy metrics. A user study assessed realism and familiarity against real-world images, while an expert comparison demonstrated significant time savings over manual scene design. The results confirm the effectiveness of multi-LLM pipelines in automating domain-specific 3D scene generation with improved reliability and precision. Future work will explore expanding the asset hierarchy, incorporating real-time generation, and adapting the pipeline to other simulation domains beyond agriculture.
Abstract (translated)
三维渲染引擎中的程序生成技术已经彻底改变了复杂环境的创建，减少了对手动设计的依赖。最近使用大型语言模型（LLMs）进行3D场景生成的方法显示出潜力，但往往缺乏领域特定推理、验证机制和模块化设计能力。这些限制导致控制减弱和可扩展性差。本文探讨了利用LLM根据自然语言提示生成农业合成仿真环境的方法，以解决缺乏领域特定推理、验证机制和模块化设计的局限性。 为此，开发了一种模块化的多LLM管道，结合3D资产检索、领域知识注入以及使用Unreal渲染引擎API进行代码生成。这产生了一个基于输入提示和领域知识的真实种植布局和环境上下文的三维环境。 为了提高准确性和可扩展性，该系统采用了一种混合策略，结合了包括少量样本提示（few-shot prompting）、增强式检索生成（Retrieval-Augmented Generation, RAG）、微调和验证在内的LLM优化技术。与单一模型相比，模块化架构允许结构化数据处理、中间验证和灵活扩展。 通过使用结构化提示和语义准确性度量对系统进行了评估。一项用户研究评估了合成环境的真实感和熟悉程度，并将其与真实世界的图像进行比较；专家对比显示该方法比手动场景设计节省大量时间。结果证实了多LLM管道在自动化特定领域的3D场景生成方面的有效性，提高了可靠性和精度。 未来的工作将探索扩展资产层次结构、结合实时生成以及适应农业以外的其他模拟领域。
URL
https://arxiv.org/abs/2602.11706
