Abstract
Humans can quickly generalize handwriting styles from a single example by intuitively separating content from style. Machines, however, struggle with this task, especially in low-data settings, often missing subtle spatial and stylistic cues. Motivated by this gap, we introduce WriteViT, a one-shot handwritten text synthesis framework that incorporates Vision Transformers (ViT), a family of models that have shown strong performance across various computer vision tasks. WriteViT integrates a ViT-based Writer Identifier for extracting style embeddings, a multi-scale generator built with Transformer encoder-decoder blocks enhanced by conditional positional encoding (CPE), and a lightweight ViT-based recognizer. While previous methods typically rely on CNNs or CRNNs, our design leverages transformers in key components to better capture both fine-grained stroke details and higher-level style information. Although handwritten text synthesis has been widely explored, its application to Vietnamese -- a language rich in diacritics and complex typography -- remains limited. Experiments on Vietnamese and English datasets demonstrate that WriteViT produces high-quality, style-consistent handwriting while maintaining strong recognition performance in low-resource scenarios. These results highlight the promise of transformer-based designs for multilingual handwriting generation and efficient style adaptation.
Abstract (translated)
人类可以从单一的样本中快速地概括出手写风格，通过直观地区分内容和风格。然而，机器在处理这种任务时，尤其是在数据量较少的情况下，经常难以捕捉到细微的空间和风格线索。为了弥补这一差距，我们引入了一种名为WriteViT的一次性手写文本合成框架，该框架结合了视觉变换器（Vision Transformers, ViT），这是一种在各种计算机视觉任务中表现出色的模型系列。WriteViT集成了基于ViT的Writer Identifier用于提取风格嵌入、采用Transformer编码-解码块并通过条件位置编码(CPE)增强的多尺度生成器，以及一个轻量级的基于ViT的识别器。相比之下，之前的方法通常依赖于CNN或CRNN模型，而我们的设计在关键组件中采用了变换器来更好地捕捉细节笔画特征和更高级别的风格信息。 尽管手写文本合成已经被广泛研究，但在越南语这种具有丰富音调符号和复杂版面的语言上的应用仍然有限。实验表明，在越南语和英语数据集上，WriteViT能够生成高质量且风格一致的手写字体，并在低资源场景中保持强大的识别性能。这些结果突显了基于变换器的设计在多语言手写生成和高效风格适应方面的潜力。
URL
https://arxiv.org/abs/2505.13235
