Abstract
Existing scene text spotters are designed to locate and transcribe texts from images. However, it is challenging for a spotter to achieve precise detection and recognition of scene texts simultaneously. Inspired by the glimpse-focus spotting pipeline of human beings and impressive performances of Pre-trained Language Models (PLMs) on visual tasks, we ask: 1) "Can machines spot texts without precise detection just like human beings?", and if yes, 2) "Is text block another alternative for scene text spotting other than word or character?" To this end, our proposed scene text spotter leverages advanced PLMs to enhance performance without fine-grained detection. Specifically, we first use a simple detector for block-level text detection to obtain rough positional information. Then, we finetune a PLM using a large-scale OCR dataset to achieve accurate recognition. Benefiting from the comprehensive language knowledge gained during the pre-training phase, the PLM-based recognition module effectively handles complex scenarios, including multi-line, reversed, occluded, and incomplete-detection texts. Taking advantage of the fine-tuned language model on scene recognition benchmarks and the paradigm of text block detection, extensive experiments demonstrate the superior performance of our scene text spotter across multiple public benchmarks. Additionally, we attempt to spot texts directly from an entire scene image to demonstrate the potential of PLMs, even Large Language Models (LLMs).
Abstract (translated)
现有的场景文本检测器旨在从图像中定位和转录文本。然而，对于检测器来说，同时实现精确的检测和识别场景文本是非常具有挑战性的。受到人类瞥视焦点检测流程和预训练语言模型（PLMs）在视觉任务上的出色表现启发，我们提出以下问题：1）“机器能否像人类一样准确地检测到文本，而无需进行精确的检测？”如果答案是肯定的，2）“文本块是否是场景文本检测的另一种选择，除了单词或字符？”为此，我们提出的场景文本检测器利用预训练语言模型（PLMs）增强性能，而无需进行微细检测。具体来说，我们首先使用简单的基于块级的文本检测器获得粗略的位置信息。然后，我们使用一个大型的OCR数据集对PLM进行微调，以实现精确的识别。得益于在预训练阶段获得的全面语言知识，基于PLM的识别模块有效地处理复杂的场景，包括多行、反向、遮挡和不完整的检测文本。利用场景识别基准测试中的微调后的语言模型以及文本块检测的范式，广泛的实验证明了我们在多个公共基准测试中的优越性能。此外，我们还尝试从整个场景图像中直接检测文本，以展示PLMs的潜力，即使是大语言模型（LLMs）。
URL
https://arxiv.org/abs/2403.10047
