Abstract
Large Vision-Language Models have achieved significant reasoning performance in various this http URL, there are few studies on text-to-3D indoor scene generation with LVLMs. The main challenge is that prevailing LVLM-based methods employ chain-of-thought sequential decision mechanisms that cannot revise earlier decisions, causing error this http URL this paper, we consider the task as a planning problem constrained by spatial and layout this http URL solve this problem, we model it as a tree search problem with global and local trees, which differs from existing sequential decision-making this http URL the global tree, we place each object iteratively and explore multiple attempts like humans furnishing a room, where the problem space is represented as a this http URL effectively search the tree, we propose a hierarchical scene representation and a PRM-guided MCTS this http URL hierarchical representation abstracts a scene into room level, region level, floor object level, and supported object this http URL PRM-guided MCTS method uses the PRM to prune unnecessary branches and the MCTS algorithm to balance exploration and exploitation to get an optimal solution with fewer this http URL the local tree, it further decomposes the placement of each object into finer sub-steps, including the specific placement this http URL make the whole appearance of the scene consistent, we leverage pre-trained diffusion image generative models to predict textures for all the objects in the this http URL existing benchmarks for text-to-3D indoor scene generation remain limited in scale and diversity, we collect a new large-scale diverse dataset that contains 65 scene types and 3,250 instructions with diverse sizes, layouts, and styles, named 3DTindo-bench, to better assess the capability of the state-of-the-art models. Our experiments show that our method generates more realistic 3D scenes than state-of-the-art approaches.
Abstract (translated)
URL
https://arxiv.org/abs/2606.06002
