Abstract
Handwritten text recognition aims to convert visual input into machine-readable text, and it remains challenging due to the evolving and context-dependent nature of handwriting. Character sets change over time, and character frequency distributions shift across historical periods or regions, often causing models trained on broad, heterogeneous corpora to underperform on specific subsets. To tackle this, we propose a novel loss function that incorporates the Wasserstein distance between the character frequency distribution of the predicted text and a target distribution empirically derived from training data. By penalizing divergence from expected distributions, our approach enhances both accuracy and robustness under temporal and contextual intra-dataset shifts. Furthermore, we demonstrate that character distribution alignment can also improve existing models at inference time without requiring retraining by integrating it as a scoring function in a guided decoding scheme. Experimental results across multiple datasets and architectures confirm the effectiveness of our method in boosting generalization and performance. We open source our code at this https URL.
Abstract (translated)
手写文本识别旨在将视觉输入转换为机器可读的文本，但由于手写的演变和上下文依赖性，这一过程仍然具有挑战性。字符集会随时间变化，不同历史时期或地区的字符频率分布也会有所不同，这通常会导致在广泛、异质语料库上训练的模型在其特定子集中表现不佳。为此，我们提出了一种新的损失函数，该函数结合了预测文本中的字符频率分布与从训练数据中经验性推导出的目标分布之间的Wasserstein距离。通过惩罚偏离预期分布的行为，我们的方法可以提高模型在时间变化和上下文内的数据集内部变化情况下的准确性和鲁棒性。此外，我们还证明，在推理阶段将字符分布对齐整合为指导解码方案中的评分函数，可以在不重新训练的情况下改善现有模型的性能。通过多个数据集和架构上的实验结果证实了我们的方法在提高泛化能力和性能方面的有效性。我们将源代码开源于此 [URL]（请用实际链接替换）。
URL
https://arxiv.org/abs/2506.09846
