Abstract
In recent years, the optical character recognition (OCR) field has been proliferating with plentiful cutting-edge approaches for a wide spectrum of tasks. However, these approaches are task-specifically designed with divergent paradigms, architectures, and training strategies, which significantly increases the complexity of research and maintenance and hinders the fast deployment in applications. To this end, we propose UPOCR, a simple-yet-effective generalist model for Unified Pixel-level OCR interface. Specifically, the UPOCR unifies the paradigm of diverse OCR tasks as image-to-image transformation and the architecture as a vision Transformer (ViT)-based encoder-decoder. Learnable task prompts are introduced to push the general feature representations extracted by the encoder toward task-specific spaces, endowing the decoder with task awareness. Moreover, the model training is uniformly aimed at minimizing the discrepancy between the generated and ground-truth images regardless of the inhomogeneity among tasks. Experiments are conducted on three pixel-level OCR tasks including text removal, text segmentation, and tampered text detection. Without bells and whistles, the experimental results showcase that the proposed method can simultaneously achieve state-of-the-art performance on three tasks with a unified single model, which provides valuable strategies and insights for future research on generalist OCR models. Code will be publicly available.
Abstract (translated)
近年来，光学字符识别（OCR）领域已经出现了许多丰富的新方法，用于各种任务。然而，这些方法都是针对特定任务设计的具有分叉范式、架构和训练策略的，这使得研究和维护变得复杂，并阻碍了在应用中快速部署。为此，我们提出了UPOCR，一种简单而有效的统一像素级OCR接口通用模型。具体来说，UPOCR将多样OCR任务的范式统一为图像到图像变换，将架构统一为基于视觉Transformer（ViT）的编码器-解码器。引入了可学习任务提示，将编码器提取的通用特征表示推向任务特定空间，使解码器具有任务意识。此外，模型训练目标是在不同任务间保持生成图像和真实图像之间的差异最小化。实验对包括文本去除、文本分割和篡改文本检测在内的三个像素级OCR任务进行了研究。没有花言巧语，实验结果表明，所提出的统一单模型可以同时实现三个任务的最先进性能，该模型为未来研究提供了有价值的策略和见解。代码将公开发布。
URL
https://arxiv.org/abs/2312.02694
