Abstract
AI-assisted graphic design has emerged as a powerful tool for automating the creation and editing of design elements such as posters, banners, and advertisements. While diffusion-based text-to-image models have demonstrated strong capabilities in visual content generation, their text rendering performance, particularly for small-scale typography and non-Latin scripts, remains limited. In this paper, we propose UTDesign, a unified framework for high-precision stylized text editing and conditional text generation in design images, supporting both English and Chinese scripts. Our framework introduces a novel DiT-based text style transfer model trained from scratch on a synthetic dataset, capable of generating transparent RGBA text foregrounds that preserve the style of reference glyphs. We further extend this model into a conditional text generation framework by training a multi-modal condition encoder on a curated dataset with detailed text annotations, enabling accurate, style-consistent text synthesis conditioned on background images, prompts, and layout specifications. Finally, we integrate our approach into a fully automated text-to-design (T2D) pipeline by incorporating pre-trained text-to-image (T2I) models and an MLLM-based layout planner. Extensive experiments demonstrate that UTDesign achieves state-of-the-art performance among open-source methods in terms of stylistic consistency and text accuracy, and also exhibits unique advantages compared to proprietary commercial approaches. Code and data for this paper are available at this https URL.
Abstract (translated)
AI辅助的图形设计已作为自动化创建和编辑如海报、横幅和广告等设计元素的强大工具兴起。尽管基于扩散模型的文本到图像生成方法在视觉内容生成方面展示了强大的能力，但它们在小规模排版和非拉丁文字渲染上的性能仍然有限。本文中，我们提出了UTDesign框架，这是一个用于高精度风格化文本编辑及条件文本生成的设计图象统一框架，支持英文和中文脚本。我们的框架引入了一个基于DiT的新型文本样式迁移模型，该模型从合成数据集上训练得出，能够生成保持参考字符样式的透明RGBA文本前景。我们进一步通过在精心策划并带有详细文本注释的数据集上训练一个多模态条件编码器，将此模型扩展为一个条件文本生成框架，使基于背景图像、提示和布局规范的准确且风格一致的文字合成成为可能。最后，我们将方法整合到完全自动化的文本到设计（Text-to-Design, T2D）工作流中，通过融合预训练的文本到图像（T2I）模型及MLLM布局规划器实现这一目标。广泛的实验表明，UTDesign在开源方法中达到了风格一致性和文字准确性方面的顶尖水平，并且相比专有的商业方法也展现出独特的优势。本文代码和数据可在提供的链接获取。
URL
https://arxiv.org/abs/2512.20479
