Abstract
Recognizing text lines from images is a challenging problem, especially for handwritten documents due to large variations in writing styles. While text line recognition models are generally trained on large corpora of real and synthetic data, such models can still make frequent mistakes if the handwriting is inscrutable or the image acquisition process adds corruptions, such as noise, blur, compression, etc. Writing style is generally quite consistent for an individual, which can be leveraged to correct mistakes made by such models. Motivated by this, we introduce the problem of adapting text line recognition models during test time. We focus on a challenging and realistic setting where, given only a single test image consisting of multiple text lines, the task is to adapt the model such that it performs better on the image, without any labels. We propose an iterative self-training approach that uses feedback from the language model to update the optical model, with confident self-labels in each iteration. The confidence measure is based on an augmentation mechanism that evaluates the divergence of the prediction of the model in a local region. We perform rigorous evaluation of our method on several benchmark datasets as well as their corrupted versions. Experimental results on multiple datasets spanning multiple scripts show that the proposed adaptation method offers an absolute improvement of up to 8% in character error rate with just a few iterations of self-training at test time.
Abstract (translated)
识别图像中的文本线条是一个挑战性的问题,特别是对于手写文档,因为书写风格有很大的差异。虽然文本线条识别模型通常是基于大量真实和合成数据的大型数据集训练的,但如果手写字迹难以辨认或图像采集过程会增加噪声、模糊、压缩等错误,这些模型仍然可能频繁犯错。书写风格通常为个人非常一致,可以利用它来纠正这些模型的错误。基于这种想法,我们提出了在测试期间适应文本线条识别模型的问题。我们关注一个具有挑战性和实际性的情境,其中给定只有一张包含多个文本线条的测试图像,任务是适应模型,使其在图像中表现更好,而不需要标签。我们提出了一种迭代的自我训练方法,使用语言模型的反馈更新光学模型,在每个迭代中都有一个自信的自我标签。信心测量基于增强机制,评估模型在局部区域的预测差异。我们对多个基准数据集以及其损坏版本进行了严格的评估。多个数据集跨越多个脚本的实验结果显示, proposed 适应方法在测试时仅需要进行几个迭代的自我训练,就能 absolute 地提高字符错误率,达到8%的水平。
URL
https://arxiv.org/abs/2308.15037
