Abstract
Scene text recognition (STR) is a challenging task that requires large-scale annotated data for training. However, collecting and labeling real text images is expensive and time-consuming, which limits the availability of real data. Therefore, most existing STR methods resort to synthetic data, which may introduce domain discrepancy and degrade the performance of STR models. To alleviate this problem, recent semi-supervised STR methods exploit unlabeled real data by enforcing character-level consistency regularization between weakly and strongly augmented views of the same image. However, these methods neglect word-level consistency, which is crucial for sequence recognition tasks. This paper proposes a novel semi-supervised learning method for STR that incorporates word-level consistency regularization from both visual and semantic aspects. Specifically, we devise a shortest path alignment module to align the sequential visual features of different views and minimize their distance. Moreover, we adopt a reinforcement learning framework to optimize the semantic similarity of the predicted strings in the embedding space. We conduct extensive experiments on several standard and challenging STR benchmarks and demonstrate the superiority of our proposed method over existing semi-supervised STR methods.
Abstract (translated)
场景文本识别（STR）是一个具有挑战性的任务，需要大量标注数据进行训练。然而，收集和标注真实文本图像成本昂贵且耗时，这限制了真实数据的可用性。因此，大多数现有的STR方法求助于合成数据，这可能导致领域不一致并降低STR模型的性能。为了减轻这个问题，最近的一些半监督STR方法利用无标签真实数据，通过在同一图像的弱化和增强视图之间强制字符级别一致性正则化来利用它们。然而，这些方法忽视了词级别一致性，这对于序列识别任务至关重要。本文提出了一种新颖的半监督STR方法，该方法从视觉和语义方面结合词级别一致性正则化。具体来说，我们设计了一个最短路径对齐模块，对不同视图的序列视觉特征进行对齐，并最小化它们的距离。此外，我们采用强化学习框架来优化在嵌入空间中预测字符的语义相似度。我们在多个标准和具有挑战性的STR基准上进行了广泛的实验，并证明了与现有半监督STR方法相比，我们提出的方法具有优越性。
URL
https://arxiv.org/abs/2402.15806
