Abstract
Text-to-texture synthesis has become a new frontier in 3D content creation thanks to the recent advances in text-to-image models. Existing methods primarily adopt a combination of pretrained depth-aware diffusion and inpainting models, yet they exhibit shortcomings such as 3D inconsistency and limited controllability. To address these challenges, we introduce InteX, a novel framework for interactive text-to-texture synthesis. 1) InteX includes a user-friendly interface that facilitates interaction and control throughout the synthesis process, enabling region-specific repainting and precise texture editing. 2) Additionally, we develop a unified depth-aware inpainting model that integrates depth information with inpainting cues, effectively mitigating 3D inconsistencies and improving generation speed. Through extensive experiments, our framework has proven to be both practical and effective in text-to-texture synthesis, paving the way for high-quality 3D content creation.
Abstract (translated)
文本到纹理合成已成为3D内容创作的新前沿，得益于最近在文本到图像模型方面的进步。现有的方法主要采用预训练的深度感知扩散和修复模型，但它们存在一些缺陷，如3D不一致性和可控制性有限。为了应对这些挑战，我们引入了InteX，一种交互式的文本到纹理合成框架。1) InteX包括一个用户友好的界面，在整个合成过程中促进交互和控制，实现区域特定的重新绘制和精确纹理编辑。2) 此外，我们开发了一个统一的深度感知修复模型，将深度信息与修复提示相结合，有效缓解3D不一致性并提高生成速度。通过大量的实验，我们的框架在文本到纹理合成上被证明既实用又有效，为高质量3D内容创作铺平道路。
URL
https://arxiv.org/abs/2403.11878
