Abstract
Recent advancements in Optical Character Recognition (OCR) have been driven by transformer-based models. OCR systems are critical in numerous high-stakes domains, yet their vulnerability to adversarial attack remains largely uncharted territory, raising concerns about security and compliance with emerging AI regulations. In this work we present a novel framework to assess the resilience of Transformer-based OCR (TrOCR) models. We develop and assess algorithms for both targeted and untargeted attacks. For the untargeted case, we measure the Character Error Rate (CER), while for the targeted case we use the success ratio. We find that TrOCR is highly vulnerable to untargeted attacks and somewhat less vulnerable to targeted attacks. On a benchmark handwriting data set, untargeted attacks can cause a CER of more than 1 without being noticeable to the eye. With a similar perturbation size, targeted attacks can lead to success rates of around $25\%$ -- here we attacked single tokens, requiring TrOCR to output the tenth most likely token from a large vocabulary.
Abstract (translated)
近年来，在自然语言处理（NLP）领域，特别是基于Transformer的模型在光学字符识别（OCR）方面的进步取得了重要突破。OCR系统在许多高风险领域至关重要，但它们对攻击的抵抗力仍然是一个未被充分探索的领域，这引发了关于安全和人工智能法规遵守方面的担忧。在这项工作中，我们提出了一个评估Transformer-based OCR（TrOCR）模型韧性的新框架。我们开发并评估了针对目标和无目标攻击的算法。对于无目标攻击，我们测量字符误差率（CER），而对于有目标攻击，我们使用成功率。我们发现，TrOCR在无目标攻击上高度脆弱，在有目标攻击上 somewhat less vulnerable。在一个人工手写数据集上进行评估，无目标攻击可能导致超过1的CER，而目标攻击可能导致约25%的成功率。在这里，我们攻击单个标记，要求TrOCR从大型词汇中输出最有可能的标记。
URL
https://arxiv.org/abs/2311.17128
