Abstract
Digital ink -- the coordinate stream captured from stylus or touch input -- lacks a unified representation. Continuous vector representations produce long sequences and suffer from training instability, while existing token representations require large vocabularies, face out-of-vocabulary issues, and underperform vectors on recognition. We propose ScribeTokens, a tokenization that decomposes pen movement into unit pixel steps. Together with two pen-state tokens, this fixed 10-token base vocabulary suffices to represent any digital ink and enables aggressive BPE compression. On handwritten text generation, ScribeTokens dramatically outperforms vectors (17.33% vs. 70.29% CER), showing tokens are far more effective for generation. On recognition, ScribeTokens is the only token representation to outperform vectors without pretraining. We further introduce next-ink-token prediction as a self-supervised pretraining strategy, which consistently improves recognition across all token-based models and accelerates convergence by up to 83x. With pretraining, ScribeTokens achieves the best recognition results across all representations on both datasets (8.27% CER on IAM, 9.83% on DeepWriting).
Abstract (translated)
数字墨水——从笔或触摸输入捕获的坐标流——缺乏统一的表现形式。连续向量表示会产生长序列并导致训练不稳定，而现有的标记表示则需要庞大的词汇表、面临词外问题，并且在识别方面不如向量表现好。我们提出了ScribeTokens，这是一种将笔迹运动分解为单像素步进的分词方法。结合两个笔状态标记，这种固定10个标记的基础词汇足以代表任何数字墨水，并能实现激进的BPE压缩。在手写文本生成任务中，ScribeTokens显著优于向量表示（CER误差率为17.33%，而向量为70.29%），表明标记对于生成过程更为有效。在识别方面，ScribeTokens是唯一一种无需预训练即能超越向量表现的标记表示形式。我们进一步引入了下一笔标记预测作为自监督预训练策略，这种方法在所有基于标记的模型中都能一致地提升识别效果，并最多可加快83倍的收敛速度。通过预训练，ScribeTokens在两个数据集的所有表示方法中均取得了最佳识别结果（IAM上的CER误差率为8.27%，DeepWriting上的为9.83%）。
URL
https://arxiv.org/abs/2603.02805
