Abstract
Scene text images contain not only style information (font, background) but also content information (character, texture). Different scene text tasks need different information, but previous representation learning methods use tightly coupled features for all tasks, resulting in sub-optimal performance. We propose a Disentangled Representation Learning framework (DARLING) aimed at disentangling these two types of features for improved adaptability in better addressing various downstream tasks (choose what you really need). Specifically, we synthesize a dataset of image pairs with identical style but different content. Based on the dataset, we decouple the two types of features by the supervision design. Clearly, we directly split the visual representation into style and content features, the content features are supervised by a text recognition loss, while an alignment loss aligns the style features in the image pairs. Then, style features are employed in reconstructing the counterpart image via an image decoder with a prompt that indicates the counterpart's content. Such an operation effectively decouples the features based on their distinctive properties. To the best of our knowledge, this is the first time in the field of scene text that disentangles the inherent properties of the text images. Our method achieves state-of-the-art performance in Scene Text Recognition, Removal, and Editing.
Abstract (translated)
场景文本图像不仅包含样式信息（字体，背景）还包含内容信息（字符，纹理）。不同的场景文本任务需要不同的信息，但之前的表现学习方法使用紧密耦合的特征来处理所有任务，导致在应对各种下游任务时性能较低。我们提出了一个解耦表示学习框架（DARLING），旨在解耦这两种类型的特征以提高在更好地解决各种下游任务时的适应性（选择您真正需要的）。具体来说，我们通过监督设计合成了一组具有相同风格的图像对，但具有不同内容的图像。根据这个数据集，我们通过监督设计解耦这两种类型的特征。显然，我们直接将视觉表示分为样式和内容特征。内容特征由文本识别损失进行监督，而风格特征通过图像解码器的提示进行对齐。然后，通过样式特征在图像对中重构对应图像。这种操作有效地将根据其独特属性解耦。据我们所知，这是场景文本领域首次解耦文本图像的固有属性。我们的方法在场景文本识别、去除和编辑方面取得了最先进的性能。
URL
https://arxiv.org/abs/2405.04377
