Abstract
Graphic design is crucial for conveying ideas and messages. Designers usually organize their work into objects, backgrounds, and vectorized text layers to simplify editing. However, this workflow demands considerable expertise. With the rise of GenAI methods, an endless supply of high-quality graphic designs in pixel format has become more accessible, though these designs often lack editability. Despite this, non-layered designs still inspire human designers, influencing their choices in layouts and text styles, ultimately guiding the creation of layered designs. Motivated by this observation, we propose Accordion, a graphic design generation framework taking the first attempt to convert AI-generated designs into editable layered designs, meanwhile refining nonsensical AI-generated text with meaningful alternatives guided by user prompts. It is built around a vision language model (VLM) playing distinct roles in three curated stages. For each stage, we design prompts to guide the VLM in executing different tasks. Distinct from existing bottom-up methods (e.g., COLE and Open-COLE) that gradually generate elements to create layered designs, our approach works in a top-down manner by using the visually harmonious reference image as global guidance to decompose each layer. Additionally, it leverages multiple vision experts such as SAM and element removal models to facilitate the creation of graphic layers. We train our method using the in-house graphic design dataset Design39K, augmented with AI-generated design images coupled with refined ground truth created by a customized inpainting model. Experimental results and user studies by designers show that Accordion generates favorable results on the DesignIntention benchmark, including tasks such as text-to-template, adding text to background, and text de-rendering, and also excels in creating design variations.
Abstract (translated)
图形设计在传达想法和信息方面至关重要。设计师通常会将他们的作品组织成对象层、背景层和矢量文本层，以此简化编辑过程。然而，这一工作流程需要相当的专业知识。随着生成人工智能（GenAI）方法的兴起，高分辨率像素格式的高质量图形设计变得更加触手可及。尽管如此，这些设计往往缺乏可编辑性。不过，非分层的设计仍然激发着人类设计师的灵感，影响他们的布局和文字风格选择，并最终指导他们创作出分层设计。基于这一观察，我们提出了Accordion框架——这是第一个尝试将AI生成的设计转化为可编辑分层设计的方法，同时根据用户的提示来优化无意义的AI生成文本，使其具有实际含义。 Accordion围绕一个视觉语言模型（VLM）构建而成，在三个精心策划的阶段中扮演不同角色。对于每个阶段，我们设计了引导性提示，帮助VLM执行不同的任务。与现有自下而上的方法（如COLE和Open-COLE）通过逐步生成元素来创建分层设计的方法不同，我们的方法采用一种从上至下的方式，即以视觉和谐的参考图像作为全局指导，将每个层次进行分解。此外，Accordion还利用多种视觉专家模型（例如SAM和去元素除除模型），从而方便图形层级的设计创作。 我们使用内部构建的Design39K图形设计数据集对方法进行了训练，并通过定制的修复完成模型生成了与AI产生的设计方案相匹配的真实地面实况标签。实验结果和设计师进行的用户研究显示，Accordion在DesignIntention基准测试中的表现良好，包括文本到模板、背景加文字以及去渲染等任务，同时在创造设计变体方面也表现出色。
URL
https://arxiv.org/abs/2507.05601
