Abstract
We present SketchGPT, a flexible framework that employs a sequence-to-sequence autoregressive model for sketch generation, and completion, and an interpretation case study for sketch recognition. By mapping complex sketches into simplified sequences of abstract primitives, our approach significantly streamlines the input for autoregressive modeling. SketchGPT leverages the next token prediction objective strategy to understand sketch patterns, facilitating the creation and completion of drawings and also categorizing them accurately. This proposed sketch representation strategy aids in overcoming existing challenges of autoregressive modeling for continuous stroke data, enabling smoother model training and competitive performance. Our findings exhibit SketchGPT's capability to generate a diverse variety of drawings by adding both qualitative and quantitative comparisons with existing state-of-the-art, along with a comprehensive human evaluation study. The code and pretrained models will be released on our official GitHub.
Abstract (translated)
我们提出了SketchGPT,一个灵活的框架,它采用序列到序列自回归模型用于草图生成和完成,以及用于草图识别的交互案例研究。通过将复杂草图映射为抽象基本结构的简化序列,我们的方法大大简化了自回归建模的输入。SketchGPT利用下一个词预测目标策略来理解草图模式,促进草图的创作和完成,并准确分类它们。这种提出的草图表示策略有助于克服连续 stroke 数据中自回归建模的现有挑战,使得模型训练更加平滑,同时实现具有竞争力的性能。我们的研究结果表明,SketchGPT通过添加定性和定量与现有最先进的水平的比较,具有生成各种不同草图的能力,以及全面的用户评估研究。代码和预训练模型将在我们的官方 GitHub 上发布。
URL
https://arxiv.org/abs/2405.03099
