Abstract
Recent advances in vision language models (VLMs) have shown strong potential for spatial reasoning and 3D scene layout generation from open-ended language instructions. However, generating layouts that are not only semantically coherent but also feasible for interaction by embodied agents remains challenging, particularly in physically constrained indoor environments. In this paper, RoboLayout is introduced as an extension of LayoutVLM that augments the original framework with agent-aware reasoning and improved optimization stability. RoboLayout integrates explicit reachability constraints into a differentiable layout optimization process, enabling the generation of layouts that are navigable and actionable by embodied agents. Importantly, the agent abstraction is not limited to a specific robot platform and can represent diverse entities with distinct physical capabilities, such as service robots, warehouse robots, humans of different age groups, or animals, allowing environment design to be tailored to the intended agent. In addition, a local refinement stage is proposed that selectively reoptimizes problematic object placements while keeping the remainder of the scene fixed, improving convergence efficiency without increasing global optimization iterations. Overall, RoboLayout preserves the strong semantic alignment and physical plausibility of LayoutVLM while enhancing applicability to agent-centric indoor scene generation, as demonstrated by experimental results across diverse scene configurations.
Abstract (translated)
最近，视觉语言模型（VLM）在从开放式语言指令生成空间推理和3D场景布局方面展示了强大的潜力。然而，创建既语义连贯又能被具身代理交互的可行布局仍然具有挑战性，尤其是在物理受限的室内环境中。在这篇论文中，介绍了一种名为RoboLayout的新方法，它是对LayoutVLM框架的一种扩展，通过加入代理感知推理和优化稳定性改进来增强原模型的功能。RoboLayout将显式的可达性约束整合进可微布局优化过程中，从而生成能够被具身代理导航并执行的场景布局。 重要的是，这种代理抽象不仅限于特定的机器人平台，还可以表示具有不同物理能力的各种实体，例如服务型机器人、仓库机器人、各年龄段的人类或动物等，从而使环境设计可以适应不同的目标代理。此外，还提出了一种局部细化阶段，在该阶段中选择性地重新优化有问题的对象位置，同时保持场景其余部分不变，从而在不增加全局优化迭代次数的情况下提高了收敛效率。 总体而言，通过跨多种不同配置的实验结果证明，RoboLayout不仅保留了LayoutVLM的强大语义对齐和物理合理性，而且还增强了其应用于代理中心视角的室内场景生成的能力。
URL
https://arxiv.org/abs/2603.05522
