Abstract
This paper presents a real-time generative drawing system that interprets and integrates both formal intent - the structural, compositional, and stylistic attributes of a sketch - and contextual intent - the semantic and thematic meaning inferred from its visual content - into a unified transformation process. Unlike conventional text-prompt-based generative systems, which primarily capture high-level contextual descriptions, our approach simultaneously analyzes ground-level intuitive geometric features such as line trajectories, proportions, and spatial arrangement, and high-level semantic cues extracted via vision-language models. These dual intent signals are jointly conditioned in a multi-stage generation pipeline that combines contour-preserving structural control with style- and content-aware image synthesis. Implemented with a touchscreen-based interface and distributed inference architecture, the system achieves low-latency, two-stage transformation while supporting multi-user collaboration on shared canvases. The resulting platform enables participants, regardless of artistic expertise, to engage in synchronous, co-authored visual creation, redefining human-AI interaction as a process of co-creation and mutual enhancement.
Abstract (translated)
本文介绍了一种实时生成绘画系统，该系统能够解读并整合两种意图：形式意图——草图的结构、构图和风格属性；以及情境意图——从视觉内容中推断出的语义和主题含义，并将它们统一到一个转换过程中。与传统的基于文本提示的生成系统主要捕捉高层次的情境描述不同，我们的方法同时分析了基础层面直观的几何特征（如线条轨迹、比例及空间布局）和通过视觉语言模型提取的高层次语义线索。这些双重意图信号在一个多阶段生成管道中联合调节，该管道结合了保持轮廓结构控制与风格和内容感知图像合成的方法。系统采用触摸屏界面和分布式推理架构实现，在支持多人协作的同时实现了低延迟、两阶段转换，并且参与者无论是否有艺术背景都可以进行同步的共同创作，重新定义了人机交互作为一种共创和相互增强的过程。
URL
https://arxiv.org/abs/2508.19254
