Abstract
We present Pro-DG, a framework for procedurally controllable photo-realistic facade generation that combines a procedural shape grammar with diffusion-based image synthesis. Starting from a single input image, we reconstruct its facade layout using grammar rules, then edit that structure through user-defined transformations. As facades are inherently multi-hierarchical structures, we introduce hierarchical matching procedure that aligns facade structures at different levels which is used to introduce control maps to guide a generative diffusion pipeline. This approach retains local appearance fidelity while accommodating large-scale edits such as floor duplication or window rearrangement. We provide a thorough evaluation, comparing Pro-DG against inpainting-based baselines and synthetic ground truths. Our user study and quantitative measurements indicate improved preservation of architectural identity and higher edit accuracy. Our novel method is the first to integrate neuro-symbolically derived shape-grammars for modeling with modern generative model and highlights the broader potential of such approaches for precise and controllable image manipulation.
Abstract (translated)
我们介绍了一种名为Pro-DG的框架，该框架用于生成程序可控的逼真建筑立面，它结合了过程形状语法和基于扩散的图像合成技术。从单张输入图片开始，我们可以使用语法规则重建其立面布局，并通过用户定义的转换编辑这一结构。鉴于立面本质上是多层次结构，我们引入了一种层次匹配过程，该过程对不同层级上的立面结构进行对齐，以用于引导生成扩散管道的控制图。这种方法在保持局部外观真实性的前提下，还能容纳大规模编辑，例如楼层复制或窗户重新排列。 我们进行了全面的评估，并将Pro-DG与基于图像修复的方法和合成的真实数据进行了比较。我们的用户研究和定量测量表明，在保留建筑身份的同时提高了编辑准确性。我们的新方法是首个集成神经符号派生形状语法用于现代生成模型建模的技术，突显了此类方法在精确且可控的图像操作方面的广泛应用潜力。
URL
https://arxiv.org/abs/2504.01571
