Optical_Character
-
Aligning What Vision-Language Models See and Perceive with Adaptive Information Flow
2026-04-17 08:07:22Chengxin Liu, Wonseok Choi, Chenshuang Zhang, Tae-Hyun OhAbstract
Vision-Language Models (VLMs) have demonstrated strong capability in a wide range of tasks such as visual recognition, document parsing, and visual grounding. Nevertheless, recent work shows that while VLMs often manage to capture the correct image region corresponding to the question, they do not necessarily produce the correct answers. In this work, we demonstrate that this misalignment could be attributed to suboptimal information flow within VLMs, where text tokens distribute too much attention to irrelevant visual tokens, leading to incorrect answers. Based on the observation, we show that modulating the information flow during inference can improve the perception capability of VLMs. The idea is that text tokens should only be associated with important visual tokens during decoding, eliminating the interference of irrelevant regions. To achieve this, we propose a token dynamics-based method to determine the importance of visual tokens, where visual tokens that exhibit distinct activation patterns during different decoding stages are viewed as important. We apply our approach to representative open-source VLMs and evaluate on various datasets, including visual question answering, visual grounding and counting, optical character recognition, and object hallucination. The results show that our approach significantly improves the performance of baselines. Project page: this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2604.15809
PDF
-
GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts
2026-04-14 17:12:41Amir Hossein Kargaran, Nafiseh Nikeghbal, Jana Diesner, Fran\c{c}ois Yvon, Hinrich Sch\"utzeAbstract
Optical character recognition (OCR) has advanced rapidly with the rise of vision-language models, yet evaluation has remained concentrated on a small cluster of high- and mid-resource scripts. We introduce GlotOCR Bench, a comprehensive benchmark evaluating OCR generalization across 100+ Unicode scripts. Our benchmark comprises clean and degraded image variants rendered from real multilingual texts. Images are rendered using fonts from the Google Fonts repository, shaped with HarfBuzz and rasterized with FreeType, supporting both LTR and RTL scripts. Samples of rendered images were manually reviewed to verify correct rendering across all scripts. We evaluate a broad suite of open-weight and proprietary vision-language models and find that most perform well on fewer than ten scripts, and even the strongest frontier models fail to generalize beyond thirty scripts. Performance broadly tracks script-level pretraining coverage, suggesting that current OCR systems rely on language model pretraining as much as on visual recognition. Models confronted with unfamiliar scripts either produce random noise or hallucinate characters from similar scripts they already know. We release the benchmark and pipeline for reproducibility. Pipeline Code: this https URL, Benchmark: this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2604.12978
PDF
-
Dual-Control Frequency-Aware Diffusion Model for Depth-Dependent Optical Microrobot Microscopy Image Generation
2026-04-13 16:24:08Lan Wei, Zongcai Tan, Kangyi Lu, Jian-Qing Zheng, Dandan ZhangAbstract
Optical microrobots actuated by optical tweezers (OT) are important for cell manipulation and microscale assembly, but their autonomous operation depends on accurate 3D perception. Developing such perception systems is challenging because large-scale, high-quality microscopy datasets are scarce, owing to complex fabrication processes and labor-intensive annotation. Although generative AI offers a promising route for data augmentation, existing generative adversarial network (GAN)-based methods struggle to reproduce key optical characteristics, particularly depth-dependent diffraction and defocus effects. To address this limitation, we propose Du-FreqNet, a dual-control, frequency-aware diffusion model for physically consistent microscopy image synthesis. The framework features two independent ControlNet branches to encode microrobot 3D point clouds and depth-specific mesh layers, respectively. We introduce an adaptive frequency-domain loss that dynamically reweights high- and low-frequency components based on the distance to the focal plane. By leveraging differentiable FFT-based supervision, Du-FreqNet captures physically meaningful frequency distributions often missed by pixel-space methods. Trained on a limited dataset (e.g., 80 images per pose), our model achieves controllable, depth-dependent image synthesis, improving SSIM by 20.7% over baselines. Extensive experiments demonstrate that Du-FreqNet generalizes effectively to unseen poses and significantly enhances downstream tasks, including 3D pose and depth estimation, thereby facilitating robust closed-loop control in microrobotic systems.
Abstract (translated)
URL
https://arxiv.org/abs/2604.11680
PDF
-
DocRevive: A Unified Pipeline for Document Text Restoration
2026-04-11 07:50:20Kunal Purkayastha, Ayan Banerjee, Josep Llados, Umapada PalAbstract
In Document Understanding, the challenge of reconstructing damaged, occluded, or incomplete text remains a critical yet unexplored problem. Subsequent document understanding tasks can benefit from a document reconstruction process. In response, this paper presents a novel unified pipeline combining state-of-the-art Optical Character Recognition (OCR), advanced image analysis, masked language modeling, and diffusion-based models to restore and reconstruct text while preserving visual integrity. We create a synthetic dataset of 30{,}078 degraded document images that simulates diverse document degradation scenarios, setting a benchmark for restoration tasks. Our pipeline detects and recognizes text, identifies degradation with an occlusion detector, and uses an inpainting model for semantically coherent reconstruction. A diffusion-based module seamlessly reintegrates text, matching font, size, and alignment. To evaluate restoration quality, we propose a Unified Context Similarity Metric (UCSM), incorporating edit, semantic, and length similarities with a contextual predictability measure that penalizes deviations when the correct text is contextually obvious. Our work advances document restoration, benefiting archival research and digital preservation while setting a new standard for text reconstruction. The OPRB dataset and code are available at \href{this https URL}{Hugging Face} and \href{this https URL}{Github} respectively.
Abstract (translated)
URL
https://arxiv.org/abs/2604.10077
PDF
-
AtlasOCR: Building the First Open-Source Darija OCR Model with Vision Language Models
2026-04-09 10:38:23Imane Momayiz, Soufiane Ait Elaouad, Abdeljalil Elmajjodi, Haitame BouananeAbstract
Darija, the Moroccan Arabic dialect, is rich in visual content yet lacks specialized Optical Character Recognition (OCR) tools. This paper introduces AtlasOCR, the first open-source Darija OCR model built by fine-tuning a 3B parameter Vision Language Model (VLM). We detail our comprehensive approach, from curating a unique Darija-specific dataset leveraging both synthetic generation with our OCRSmith library and carefully sourced real-world data, to implementing efficient fine-tuning strategies. We utilize QLoRA and Unsloth for parameter-efficient training of Qwen2.5-VL 3B and present comprehensive ablation studies optimizing key hyperparameters. Our evaluation on the newly curated AtlasOCRBench and the established KITAB-Bench demonstrates state-of-the-art performance, challenging larger models and highlighting AtlasOCR's robustness and generalization capabilities for both Darija and standard Arabic OCR tasks.
Abstract (translated)
URL
https://arxiv.org/abs/2604.08070
PDF
-
Multi-Head Attention based interaction-aware architecture for Bangla Handwritten Character Recognition: Introducing a Primary Dataset
2026-04-08 13:18:08Mirza Raquib, Asif Pervez Polok, Kedar Nath Biswas, Farida Siddiqi Prity, Saydul Akbar Murad, Nick RahimiAbstract
Character recognition is the fundamental part of an optical character recognition (OCR) system. Word recognition, sentence transcription, document digitization, and language processing are some of the higher-order activities that can be done accurately through character recognition. Nonetheless, recognizing handwritten Bangla characters is not an easy task because they are written in different styles with inconsistent stroke patterns and a high degree of visual character resemblance. The datasets available are usually limited in intra-class and inequitable in class distribution. We have constructed a new balanced dataset of Bangla written characters to overcome those problems. This consists of 78 classes and each class has approximately 650 samples. It contains the basic characters, composite (Juktobarno) characters and numerals. The samples were a diverse group comprising a large age range and socioeconomic groups. Elementary and high school students, university students, and professionals are the contributing factors. The sample also has right and left-handed writers. We have further proposed an interaction-aware hybrid deep learning architecture that integrates EfficientNetB3, Vision Transformer, and Conformer modules in parallel. A multi-head cross-attention fusion mechanism enables effective feature interaction across these components. The proposed model achieves 98.84% accuracy on the constructed dataset and 96.49% on the external CHBCR benchmark, demonstrating strong generalization capability. Grad-CAM visualizations further provide interpretability by highlighting discriminative regions. The dataset and source code of this research is publicly available at: this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2604.09717
PDF
-
LLM-based Schema-Guided Extraction and Validation of Missing-Person Intelligence from Heterogeneous Data Sources
2026-04-08 01:35:56Joshua Castillo, Ravi MukkamalaAbstract
Missing-person and child-safety investigations rely on heterogeneous case documents, including structured forms, bulletin-style posters, and narrative web profiles. Variations in layout, terminology, and data quality impede rapid triage, large-scale analysis, and search-planning workflows. This paper introduces the Guardian Parser Pack, an AI-driven parsing and normalization pipeline that transforms multi-source investigative documents into a unified, schema-compliant representation suitable for operational review and downstream spatial modeling. The proposed system integrates (i) multi-engine PDF text extraction with Optical Character Recognition (OCR) fallback, (ii) rule-based source identification with source-specific parsers, (iii) schema-first harmonization and validation, and (iv) an optional Large Language Model (LLM)-assisted extraction pathway incorporating validator-guided repair and shared geocoding services. We present the system architecture, key implementation decisions, and output design, and evaluate performance using both gold-aligned extraction metrics and corpus-level operational indicators. On a manually aligned subset of 75 cases, the LLM-assisted pathway achieved substantially higher extraction quality than the deterministic comparator (F1 = 0.8664 vs. 0.2578), while across 517 parsed records per pathway it also improved aggregate key-field completeness (96.97\% vs. 93.23\%). The deterministic pathway remained much faster (mean runtime 0.03 s/record vs. 3.95 s/record for the LLM pathway). In the evaluated run, all LLM outputs passed initial schema validation, so validator-guided repair functioned as a built-in safeguard rather than a contributor to the observed gains. These results support controlled use of probabilistic AI within a schema-first, auditable pipeline for high-stakes investigative settings.
Abstract (translated)
URL
https://arxiv.org/abs/2604.06571
PDF
-
The Character Error Vector: Decomposable errors for page-level OCR evaluation
2026-04-07 17:56:06Jonathan Bourne, Mwiza Simbeye, Joseph NockelsAbstract
The Character Error Rate (CER) is a key metric for evaluating the quality of Optical Character Recognition (OCR). However, this metric assumes that text has been perfectly parsed, which is often not the case. Under page-parsing errors, CER becomes undefined, limiting its use as a metric and making evaluating page-level OCR challenging, particularly when using data that do not share a labelling schema. We introduce the Character Error Vector (CEV), a bag-of-characters evaluator for OCR. The CEV can be decomposed into parsing and OCR, and interaction error components. This decomposability allows practitioners to focus on the part of the Document Understanding pipeline that will have the greatest impact on overall text extraction quality. The CEV can be implemented using a variety of methods, of which we demonstrate SpACER (Spatially Aware Character Error Rate) and a Character distribution method using the Jensen-Shannon Distance. We validate the CEV's performance against other metrics: first, the relationship with CER; then, parse quality; and finally, as a direct measure of page-level OCR quality. The validation process shows that the CEV is a valuable bridge between parsing metrics and local metrics like CER. We analyse a dataset of archival newspapers made of degraded images with complex layouts and find that state-of-the-art end-to-end models are outperformed by more traditional pipeline approaches. Whilst the CEV requires character-level positioning for optimal triage, thresholding on easily available values can predict the main error source with an F1 of 0.91. We provide the CEV as part of a Python library to support Document understanding research.
Abstract (translated)
字符错误率（CER）是评估光学字符识别（OCR）质量的关键指标。然而，该指标假设文本已被完美解析，而实际情况往往并非如此。在存在页面解析错误时，CER变得无法定义，这限制了其作为评估指标的应用，并使得对页面级OCR的评估变得困难，尤其是在使用标注模式不一致的数据时。我们提出了字符错误向量（CEV），一种用于OCR的字符袋评估器。CEV可分解为解析错误、OCR错误及交互误差三部分。这种可分解性使从业者能够聚焦于文档理解流水线中对整体文本提取质量影响最大的环节。CEV可通过多种方法实现，其中我们展示了空间感知字符错误率（SpACER）以及基于Jensen-Shannon距离的字符分布方法。我们通过以下方式验证CEV相对于其他指标的性能：首先，与CER的关系；其次，解析质量；最后，作为页面级OCR质量的直接度量。验证过程表明，CEV在解析指标与CER等局部指标之间建立了有价值的桥梁。我们分析了一个由布局复杂、图像退化的档案报纸组成的数据集，发现最先进的端到端模型表现不如更传统的流水线方法。尽管CEV需要字符级定位以实现最佳分类，但基于易获取的数值进行阈值设定即可以0.91的F1值预测主要误差来源。我们已将CEV作为Python库的一部分提供，以支持文档理解研究。
URL
https://arxiv.org/abs/2604.06160
PDF
-
Toward Unified Fine-Grained Vehicle Classification and Automatic License Plate Recognition
2026-04-07 00:10:56Gabriel E. Lima, Valfride Nascimento, Eduardo Santos, Eduil Nascimento Jr, Rayson Laroca, David MenottiAbstract
Extracting vehicle information from surveillance images is essential for intelligent transportation systems, enabling applications such as traffic monitoring and criminal investigations. While Automatic License Plate Recognition (ALPR) is widely used, Fine-Grained Vehicle Classification (FGVC) offers a complementary approach by identifying vehicles based on attributes such as color, make, model, and type. Although there have been advances in this field, existing studies often assume well-controlled conditions, explore limited attributes, and overlook FGVC integration with ALPR. To address these gaps, we introduce UFPR-VeSV, a dataset comprising 24,945 images of 16,297 unique vehicles with annotations for 13 colors, 26 makes, 136 models, and 14 types. Collected from the Military Police of Paraná (Brazil) surveillance system, the dataset captures diverse real-world conditions, including partial occlusions, nighttime infrared imaging, and varying lighting. All FGVC annotations were validated using license plate information, with text and corner annotations also being provided. A qualitative and quantitative comparison with established datasets confirmed the challenging nature of our dataset. A benchmark using five deep learning models further validated this, revealing specific challenges such as handling multicolored vehicles, infrared images, and distinguishing between vehicle models that share a common platform. Additionally, we apply two optical character recognition models to license plate recognition and explore the joint use of FGVC and ALPR. The results highlight the potential of integrating these complementary tasks for real-world applications. The UFPR-VeSV dataset is publicly available at: this https URL.
Abstract (translated)
从监控图像中提取车辆信息对于智能交通系统至关重要，可支持交通监控和刑事调查等应用。虽然自动车牌识别（ALPR）已广泛应用，但细粒度车辆分类（FGVC）通过识别颜色、品牌、型号和类型等属性提供了互补方法。尽管该领域已有进展，现有研究常假设条件受控、探索的属性有限，且忽视了FGVC与ALPR的整合。为弥补这些不足，我们推出UFPR-VeSV数据集，该数据集包含16,297辆独特车辆的24,945张图像，标注了13种颜色、26个品牌、136种型号和14种类型。数据集源自巴西巴拉那州军事警察监控系统，涵盖了多种真实世界场景，包括部分遮挡、夜间红外成像和光照变化。所有FGVC标注均通过车牌信息验证，同时提供了文本和角点标注。与现有数据集的定性和定量比较证实了本数据集的挑战性。使用五种深度学习模型的基准测试进一步验证了这一点，揭示了处理多色车辆、红外图像以及区分同平台车型等具体挑战。此外，我们应用两种光学字符识别模型进行车牌识别，并探索了FGVC与ALPR的联合使用。结果凸显了整合这些互补任务在现实应用中的潜力。UFPR-VeSV数据集已公开提供，地址为：此https链接。
URL
https://arxiv.org/abs/2604.05271
PDF
-
A ROS 2 Wrapper for Florence-2: Multi-Mode Local Vision-Language Inference for Robotic Systems
2026-04-01 17:29:59J. E. Dom\'inguez-VidalAbstract
Foundation vision-language models are becoming increasingly relevant to robotics because they can provide richer semantic perception than narrow task-specific pipelines. However, their practical adoption in robot software stacks still depends on reproducible middleware integrations rather than on model quality alone. Florence-2 is especially attractive in this regard because it unifies captioning, optical character recognition, open-vocabulary detection, grounding and related vision-language tasks within a comparatively manageable model size. This article presents a ROS 2 wrapper for Florence-2 that exposes the model through three complementary interaction modes: continuous topic-driven processing, synchronous service calls and asynchronous actions. The wrapper is designed for local execution and supports both native installation and Docker container deployment. It also combines generic JSON outputs with standard ROS 2 message bindings for detection-oriented tasks. A functional validation is reported together with a throughput study on several GPUs, showing that local deployment is feasible with consumer grade hardware. The repository is publicly available here: this https URL
Abstract (translated)
基础视觉语言模型正日益与机器人技术相关，因为它们能比狭窄的特定任务流水线提供更丰富的语义感知。然而，这些模型在机器人软件栈中的实际应用仍取决于可复现的中间件集成，而不仅仅是模型质量本身。在这方面，Florence-2 尤为突出，因为它以相对可控的模型规模统一了字幕生成、光学字符识别、开放词汇检测、定位及相关视觉语言任务。本文介绍了 Florence-2 的 ROS 2 封装器，该封装器通过三种互补的交互模式提供模型访问：连续主题驱动处理、同步服务调用和异步动作。该封装器设计用于本地执行，支持原生安装和 Docker 容器部署。它还将通用 JSON 输出与面向检测任务的标准 ROS 2 消息绑定相结合。文章报告了功能验证及在多个 GPU 上的吞吐量研究，表明在消费级硬件上实现本地部署是可行的。代码仓库已公开，链接如下：此 https URL
URL
https://arxiv.org/abs/2604.01179
PDF
-
When VLMs 'Fix' Students: Identifying and Penalizing Over-Correction in the Evaluation of Multi-line Handwritten Math OCR
2026-04-01 05:27:52Jin Seong, Wencke Liermann, Minho Kim, Jong-hun Shin, Soojong LimAbstract
Accurate transcription of handwritten mathematics is crucial for educational AI systems, yet current benchmarks fail to evaluate this capability properly. Most prior studies focus on single-line expressions and rely on lexical metrics such as BLEU, which fail to assess the semantic reasoning across multi-line student solutions. In this paper, we present the first systematic study of multi-line handwritten math Optical Character Recognition (OCR), revealing a critical failure mode of Vision-Language Models (VLMs): over-correction. Instead of faithfully transcribing a student's work, these models often "fix" errors, thereby hiding the very mistakes an educational assessment aims to detect. To address this, we propose PINK (Penalized INK-based score), a semantic evaluation metric that leverages a Large Language Model (LLM) for rubric-based grading and explicitly penalizes over-correction. Our comprehensive evaluation of 15 state-of-the-art VLMs on the FERMAT dataset reveals substantial ranking reversals compared to BLEU: models like GPT-4o are heavily penalized for aggressive over-correction, whereas Gemini 2.5 Flash emerges as the most faithful transcriber. Furthermore, human expert studies show that PINK aligns significantly better with human judgment (55.0% preference over BLEU's 39.5%), providing a more reliable evaluation framework for handwritten math OCR in educational settings.
Abstract (translated)
URL
https://arxiv.org/abs/2604.22774
PDF
-
Q-Mask: Query-driven Causal Masks for Text Anchoring in OCR-Oriented Vision-Language Models
2026-03-31 19:09:55Longwei Xu, Feng Feng, Shaojie Zhang, Xin Chen, Hang Li, Anan Du, Hailong Yu, Pei Fu, Zhenbo Luo, Jian LuanAbstract
Optical Character Recognition (OCR) is increasingly regarded as a foundational capability for modern vision-language models (VLMs), enabling them not only to read text in images but also to support downstream reasoning in real-world visual question answering (VQA). However, practical applications further require reliable text anchors, i.e., accurately grounding queried text to its corresponding spatial region. To systematically evaluate this capability, we introduce TextAnchor-Bench (TABench), a benchmark for fine-grained text-region grounding, which reveals that both general-purpose and OCR-specific VLMs still struggle to establish accurate and stable text anchors. To address this limitation, we propose Q-Mask, a precise OCR framework built upon a causal query-driven mask decoder (CQMD). Inspired by chain-of-thought reasoning, Q-Mask performs causal visual decoding that sequentially generates query-conditioned visual masks before producing the final OCR output. This visual CoT paradigm disentangles where the text is from what the text is, enforcing grounded evidence acquisition prior to recognition and enabling explicit text anchor construction during inference. To train CQMD, we construct TextAnchor-26M, a large-scale dataset of image-text pairs annotated with fine-grained masks corresponding to specific textual elements, encouraging stable text-region correspondences and injecting strong spatial priors into VLM training. Extensive experiments demonstrate that Q-Mask substantially improves text anchoring and understanding across diverse visual scenes.
Abstract (translated)
光学字符识别（OCR）正日益被视为现代视觉语言模型（VLM）的基础能力，它不仅使模型能够读取图像中的文本，还能支持现实世界视觉问答（VQA）中的下游推理。然而，实际应用进一步要求可靠的文本锚点，即准确地将查询文本定位到其对应的空间区域。为系统评估这一能力，我们推出了TextAnchor-Bench（TABench），一个用于细粒度文本-区域定位的基准测试，该测试揭示通用型和OCR专用型VLM仍难以建立准确稳定的文本锚点。为应对这一局限，我们提出了Q-Mask，一个基于因果查询驱动掩码解码器（CQMD）构建的精确OCR框架。受思维链推理启发，Q-Mask执行因果视觉解码，在生成最终OCR输出前，依次生成查询条件化的视觉掩码。这种视觉思维链范式将“文本位置”与“文本内容”解耦，强制在识别前获取有根据的证据，并在推理过程中实现显式的文本锚点构建。为训练CQMD，我们构建了TextAnchor-26M，一个大规模图像-文本对数据集，其中标注了与特定文本元素对应的细粒度掩码，以促进稳定的文本-区域对应关系，并将强空间先验注入VLM训练。大量实验表明，Q-Mask在多样化的视觉场景中显著提升了文本锚定与理解能力。
URL
https://arxiv.org/abs/2604.00161
PDF
-
Conditional Polarization Guidance for Camouflaged Object Detection
2026-03-31 17:03:06QIfan Zhang, Hao Wang, Xiangrong Qin, Ruijie LiAbstract
Camouflaged object detection (COD) aims to identify targets that are highly blended with their backgrounds. Recent works have shown that the optical characteristics of polarization cues play a significant role in improving camouflaged object detection. However, most existing polarization-based approaches depend on complex visual encoders and fusion mechanisms, leading to increased model complexity and computational overhead, while failing to fully explore how polarization can explicitly guide hierarchical RGB representation learning. To address these limitations, we propose CPGNet, an asymmetric RGB-polarization framework that introduces a conditional polarization guidance mechanism to explicitly regulate RGB feature learning for camouflaged object detection. Specifically, we design a lightweight polarization interaction module that jointly models these complementary cues and generates reliable polarization guidance in a unified manner. Unlike conventional feature fusion strategies, the proposed conditional guidance mechanism dynamically modulates RGB features using polarization priors, enabling the network to focus on subtle discrepancies between camouflaged objects and their backgrounds. Furthermore, we introduce a polarization edge-guided frequency refinement strategy that enhances high-frequency components under polarization constraints, effectively breaking camouflage patterns. Finally, we develop an iterative feedback decoder to perform coarse-to-fine feature calibration and progressively refine camouflage prediction. Extensive experiments on polarization datasets across multiple tasks, along with evaluations on non-polarization datasets, demonstrate that CPGNet consistently outperforms state-of-the-art methods.
Abstract (translated)
伪装目标检测（COD）旨在识别与背景高度融合的目标。近期研究表明，偏振线索的光学特性对提升伪装目标检测效果具有重要作用。然而，现有基于偏振的方法大多依赖复杂的视觉编码器和融合机制，导致模型复杂度与计算开销增加，且未能充分探索偏振如何显式指导分层RGB表征学习。针对这些局限，我们提出CPGNet——一种RGB-偏振非对称框架，通过引入条件偏振引导机制显式调控RGB特征学习以优化伪装目标检测。具体而言，我们设计了轻量级偏振交互模块，统一建模互补线索并生成可靠的偏振引导信号。与传统特征融合策略不同，该条件引导机制利用偏振先验动态调制RGB特征，使网络聚焦伪装目标与背景间的细微差异。此外，我们提出偏振边缘引导的频率细化策略，在偏振约束下增强高频分量以有效打破伪装模式。最后，我们开发了迭代反馈解码器，通过粗到细的特征校准逐步优化伪装预测。在跨任务偏振数据集上的大量实验及非偏振数据集评估表明，CPGNet持续优于最先进方法。
URL
https://arxiv.org/abs/2603.30008
PDF
-
L-ReLF: A Framework for Lexical Dataset Creation
2026-03-31 07:19:00Anass Sedrati, Mounir Afifi, Reda BenkhadraAbstract
This paper introduces the L-ReLF (Low-Resource Lexical Framework), a novel, reproducible methodology for creating high-quality, structured lexical datasets for underserved languages. The lack of standardized terminology, exemplified by Moroccan Darija, poses a critical barrier to knowledge equity in platforms like Wikipedia, often forcing editors to rely on inconsistent, ad-hoc methods to create new words in their language. Our research details the technical pipeline developed to overcome these challenges. We systematically address the difficulties of working with low-resource data, including source identification, utilizing Optical Character Recognition (OCR) despite its bias towards Modern Standard Arabic, and rigorous post-processing to correct errors and standardize the data model. The resulting structured dataset is fully compatible with Wikidata Lexemes, serving as a vital technical resource. The L-ReLF methodology is designed for generalizability, offering other language communities a clear path to build foundational lexical data for downstream NLP applications, such as Machine Translation and morphological analysis.
Abstract (translated)
本文介绍了L-ReLF（低资源词汇框架），这是一种新颖且可复现的方法论，旨在为资源匮乏的语言创建高质量、结构化的词汇数据集。以摩洛哥达里贾语为例，缺乏标准化术语的现象严重阻碍了维基百科等平台上的知识公平性，往往迫使编辑者依赖不一致的临时方法来为本语言创造新词。本研究详细阐述了为应对这些挑战而开发的技术流程。我们系统性地解决了处理低资源数据的困难，包括来源识别、在光学字符识别（OCR）对现代标准阿拉伯语存在偏见的情况下仍加以利用，以及通过严格的后处理来纠正错误并标准化数据模型。最终生成的结构化数据集与Wikidata Lexemes完全兼容，作为一项重要的技术资源。L-ReLF方法论设计具有通用性，为其他语言社区构建面向下游自然语言处理应用（如机器翻译和形态分析）的基础词汇数据提供了清晰路径。
URL
https://arxiv.org/abs/2603.29346
PDF
-
Transcription and Recognition of Italian Parliamentary Speeches Using Vision-Language Models
2026-03-30 07:06:49Luigi Curini, Alfio Ferrara, Giovanni Pagano, Sergio PicasciaAbstract
Parliamentary proceedings represent a rich yet challenging resource for computational analysis, particularly when preserved only as scanned historical documents. Existing efforts to transcribe Italian parliamentary speeches have relied on traditional Optical Character Recognition pipelines, resulting in transcription errors and limited semantic annotation. In this paper, we propose a pipeline based on Vision-Language Models for the automatic transcription, semantic segmentation, and entity linking of Italian parliamentary speeches. The pipeline employs a specialised OCR model to extract text while preserving reading order, followed by a large-scale Vision-Language Model that performs transcription refinement, element classification, and speaker identification by jointly reasoning over visual layout and textual content. Extracted speakers are then linked to the Chamber of Deputies knowledge base through SPARQL queries and a multi-strategy fuzzy matching procedure. Evaluation against an established benchmark demonstrates substantial improvements both in transcription quality and speaker tagging.
Abstract (translated)
URL
https://arxiv.org/abs/2603.28103
PDF
-
Efficient Domain Adaptation for Text Line Recognition via Decoupled Language Models
2026-03-30 04:39:26Arundhathi Dev, Justin ZhanAbstract
Optical character recognition remains critical infrastructure for document digitization, yet state-of-the-art performance is often restricted to well-resourced institutions by prohibitive computational barriers. End-to-end transformer architectures achieve strong accuracy but demand hundreds of GPU hours for domain adaptation, limiting accessibility for practitioners and digital humanities scholars. We present a modular detection-and-correction framework that achieves near-SOTA accuracy with single-GPU training. Our approach decouples lightweight visual character detection (domain-agnostic) from domain-specific linguistic correction using pretrained sequence models including T5, ByT5, and BART. By training the correctors entirely on synthetic noise, we enable annotation-free domain adaptation without requiring labeled target images. Evaluating across modern clean handwriting, cursive script, and historical documents, we identify a critical "Pareto frontier" in architecture selection: T5-Base excels on modern text with standard vocabulary, whereas ByT5-Base dominates on historical documents by reconstructing archaic spellings at the byte level. Our results demonstrate that this decoupled paradigm matches end-to-end transformer accuracy while reducing compute by approximately 95%, establishing a viable, resource-efficient alternative to monolithic OCR architectures.
Abstract (translated)
光学字符识别仍是文档数字化的重要基础设施，然而最先进的性能往往因高昂的计算门槛而局限于资源充足的机构。端到端Transformer架构虽能实现高精度，但领域自适应需消耗数百GPU小时，限制了实践者与数字人文研究者的使用。我们提出一种模块化的检测-校正框架，仅用单GPU训练即可实现接近SOTA的准确率。该方法将轻量级的视觉字符检测（领域无关）与基于预训练序列模型（包括T5、ByT5和BART）的领域特定语言校正解耦。通过完全在合成噪声数据上训练校正器，我们实现了无需标注目标图像的免标注领域自适应。在现代清晰手写体、草书及历史文档的跨领域评估中，我们识别出架构选择的关键"帕累托前沿"：T5-Base模型在标准词汇的现代文本上表现卓越，而ByT5-Base则凭借字节级重建古旧拼写，在历史文档中占据优势。结果表明，这种解耦范式在匹配端到端Transformer精度的同时，将计算量降低约95%，为整体式OCR架构提供了可行且资源高效的替代方案。
URL
https://arxiv.org/abs/2603.28028
PDF
-
A Survey of OCR Evaluation Methods and Metrics and the Invisibility of Historical Documents
2026-03-26 02:52:28Fitsum Sileshi Beyene, Christopher L. DancyAbstract
Optical character recognition (OCR) and document understanding systems increasingly rely on large vision and vision-language models, yet evaluation remains centered on modern, Western, and institutional documents. This emphasis masks system behavior in historical and marginalized archives, where layout, typography, and material degradation shape interpretation. This study examines how OCR and document understanding systems are evaluated, with particular attention to Black historical newspapers. We review OCR and document understanding papers, as well as benchmark datasets, which are published between 2006 and 2025 using the PRISMA framework. We look into how the studies report training data, benchmark design, and evaluation metrics for vision transformer and multimodal OCR systems. During the review, we found that Black newspapers and other community-produced historical documents rarely appear in reported training data or evaluation benchmarks. Most evaluations emphasize character accuracy and task success on modern layouts. They rarely capture structural failures common in historical newspapers, including column collapse, typographic errors, and hallucinated text. To put these findings into perspective, we use previous empirical studies and archival statistics from significant Black press collections to show how evaluation gaps lead to structural invisibility and representational harm. We propose that these gaps occur due to organizational (meso) and institutional (macro) behaviors and structure, shaped by benchmark incentives and data governance decisions.
Abstract (translated)
光学字符识别（OCR）与文档理解系统日益依赖大规模视觉及视觉-语言模型，但现有评估仍集中于当代、西方及机构文档。这种侧重掩盖了系统在历史档案与边缘化文献中的表现——在这些场景中，版式、字体与材质退化会显著影响解读。本研究聚焦OCR与文档理解系统的评估方式，特别关注黑人历史报纸。我们采用PRISMA框架，系统回顾了2006至2025年间发表的OCR与文档理解论文及基准数据集，分析其中关于视觉Transformer与多模态OCR系统的训练数据、基准设计及评估指标报告方式。综述发现，黑人报纸及其他社群制作的历史文档极少出现在已报道的训练数据或评估基准中。多数评估强调现代版式下的字符准确率与任务成功率，却很少捕捉历史报纸常见的结构性问题，如栏目塌陷、字体错误与幻觉文本。为阐释这些发现，我们结合既往实证研究与重要黑人报刊收藏的档案统计，揭示评估差距如何导致结构性隐形与表征性伤害。我们认为此类差距源于组织（中观）与制度（宏观）层面的行为与结构，其形成受基准激励与数据治理决策共同塑造。
URL
https://arxiv.org/abs/2603.25761
PDF
-
MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decoding
2026-03-23 18:25:17Hejun Dong, Junbo Niu, Bin Wang, Weijun Zeng, Wentao Zhang, Conghui HeAbstract
Optical character recognition (OCR) has evolved from line-level transcription to structured document parsing, requiring models to recover long-form sequences containing layout, tables, and formulas. Despite recent advances in vision-language models, most existing systems rely on autoregressive decoding, which introduces sequential latency and amplifies error propagation in long documents. In this work, we revisit document OCR from an inverse rendering perspective, arguing that left-to-right causal generation is an artifact of serialization rather than an intrinsic property of the task. Motivated by this insight, we propose MinerU-Diffusion, a unified diffusion-based framework that replaces autoregressive sequential decoding with parallel diffusion denoising under visual conditioning. MinerU-Diffusion employs a block-wise diffusion decoder and an uncertainty-driven curriculum learning strategy to enable stable training and efficient long-sequence inference. Extensive experiments demonstrate that MinerU-Diffusion consistently improves robustness while achieving up to 3.2x faster decoding compared to autoregressive baselines. Evaluations on the proposed Semantic Shuffle benchmark further confirm its reduced dependence on linguistic priors and stronger visual OCR capability.
Abstract (translated)
光学字符识别（OCR）已从逐行转录演进为结构化文档解析，需要模型重建包含版式、表格和公式的长序列内容。尽管视觉语言模型近期取得进展，但现有系统大多依赖自回归解码，该方法会引入顺序延迟并加剧长文档中的错误传播。本研究从逆向渲染视角重新审视文档OCR，认为从左至右的因果生成是序列化的产物而非任务的内在属性。受此启发，我们提出MinerU-Diffusion——一个基于扩散模型的统一框架，用视觉条件下的并行扩散去噪取代自回归序列解码。该方法采用分块扩散解码器与不确定性驱动的课程学习策略，实现稳定训练与高效长序列推理。大量实验表明，MinerU-Diffusion在保持鲁棒性的同时，解码速度较自回归基线提升最高达3.2倍。在提出的语义打乱基准测试中，该方法进一步验证了其对语言先验依赖的降低及更强的视觉OCR能力。
URL
https://arxiv.org/abs/2603.22458
PDF
-
DualTSR: Unified Dual-Diffusion Transformer for Scene Text Image Super-Resolution
2026-03-15 03:50:47Axi Niu, Kang Zhang, Qingsen Yan, Hao Jin, Jinqiu Sun, Yanning ZhangAbstract
Scene Text Image Super-Resolution (STISR) aims to restore high-resolution details in low-resolution text images, which is crucial for both human readability and machine recognition. Existing methods, however, often depend on external Optical Character Recognition (OCR) models for textual priors or rely on complex multi-component architectures that are difficult to train and reproduce. In this paper, we introduce DualTSR, a unified end-to-end framework that addresses both issues. DualTSR employs a single multimodal transformer backbone trained with a dual diffusion objective. It simultaneously models the continuous distribution of high-resolution images via Conditional Flow Matching and the discrete distribution of textual content via discrete diffusion. This shared design enables visual and textual information to interact at every layer, allowing the model to infer text priors internally instead of relying on an external OCR module. Compared with prior multi-branch diffusion systems, DualTSR offers a simpler end-to-end formulation with fewer hand-crafted components. Experiments on synthetic Chinese benchmarks and a curated real-world evaluation protocol show that DualTSR achieves strong perceptual quality and text fidelity.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14207
PDF
-
OSM-based Domain Adaptation for Remote Sensing VLMs
2026-03-12 11:08:30Stefan Maria Ailuro (INSAIT, Sofia University "St. Kliment Ohridski"), Mario Markov (INSAIT, Sofia University "St. Kliment Ohridski"), Mohammad Mahdi (INSAIT, Sofia University "St. Kliment Ohridski"), Delyan Boychev (INSAIT, Sofia University "St. Kliment Ohridski"), Luc Van Gool (INSAIT, Sofia University "St. Kliment Ohridski"), Danda Pani Paudel (INSAIT, Sofia University "St. Kliment Ohridski")Abstract
Vision-Language Models (VLMs) adapted to remote sensing rely heavily on domain-specific image-text supervision, yet high-quality annotations for satellite and aerial imagery remain scarce and expensive to produce. Prevailing pseudo-labeling pipelines address this gap by distilling knowledge from large frontier models, but this dependence on large teachers is costly, limits scalability, and caps achievable performance at the ceiling of the teacher. We propose OSMDA: a self-contained domain adaptation framework that eliminates this dependency. Our key insight is that a capable base VLM can serve as its own annotation engine: by pairing aerial images with rendered OpenStreetMap (OSM) tiles, we leverage optical character recognition and chart comprehension capabilities of the model to generate captions enriched by OSM's vast auxiliary metadata. The model is then fine-tuned on the resulting corpus with satellite imagery alone, yielding OSMDA-VLM, a domain-adapted VLM that requires no manual labeling and no stronger external model. We conduct exhaustive evaluations spanning 10 benchmarks across image-text-to-text tasks and comparing against 9 competitive baselines. When equally mixed with real data, our method achieves state-of-the-art results, while being substantially cheaper to train than teacher-dependent alternatives. These results suggest that, given a strong foundation model, alignment with crowd-sourced geographic data is a practical and scalable path towards remote sensing domain adaptation. Dataset and model weights will be made publicly available.
Abstract (translated)
视觉-语言模型（VLMs）在遥感领域中的应用高度依赖于特定领域的图像文本监督，然而，高质量的卫星和航空影像标注仍然稀缺且昂贵。现有的伪标签流水线通过从大型前沿模型中提取知识来弥补这一缺口，但这种对大型教师模型的依赖既成本高昂又限制了可扩展性，并将性能上限锁定在教师模型的能力水平上。为此，我们提出了OSMDA：一种自我封闭领域的适应框架，以消除这种依赖性。 我们的关键见解是，有能力的基础VLM可以充当自己的标注引擎：通过将航空影像与渲染的OpenStreetMap（OSM）瓦片配对使用，我们可以利用模型的光学字符识别和图表理解能力生成包含OSM丰富辅助元数据的说明。然后，该模型仅基于卫星图像对其进行微调，从而得到OSMDA-VLM，这是一种无需人工标注且不需要更强大外部模型的领域适应VLM。 我们进行了详尽的评估，涵盖10个跨图-文到文本任务的基准测试，并与9种竞争基线进行比较。当混合使用真实数据时，我们的方法实现了最先进的结果，同时训练成本远低于依赖于教师模型的方法。这些结果表明，在一个强大的基础模型的基础上，与众包地理信息对齐是一种实用且可扩展地走向遥感领域适应的路径。 我们将公开发布该方法的数据集和模型权重。
URL
https://arxiv.org/abs/2603.11804
PDF
