Optical_Character
-
CC-OCR V2: Benchmarking Large Multimodal Models for Literacy in Real-world Document Processing
2026-05-05 15:56:12Zhipeng Xu, Junhao Ji, Zulong Chen, Zhenghao Liu, Qing Liu, Chunyi Peng, Zubao Qin, Ze Xu, Jianqiang Wan, Jun Tang, Zhibo Yang, Shuai Bai, Dayiheng LiuAbstract
Large Multimodal Models (LMMs) have recently shown strong performance on Optical Character Recognition (OCR) tasks, demonstrating their promising capability in document literacy. However, their effectiveness in real-world applications remains underexplored, as existing benchmarks adopt task scopes misaligned with practical applications and assume homogeneous acquisition conditions. To address this gap, we introduce CC-OCR V2, a comprehensive and challenging OCR benchmark tailored to real-world document processing. CC-OCR V2 focuses on practical enterprise document processing tasks and incorporates hard and corner cases that are critical yet underrepresented in prior benchmarks, covering 5 major OCR-centric tracks: text recognition, document parsing, document grounding, key information extraction, and document question answering, comprising 7,093 high-difficulty samples. Extensive experiments on 14 advanced LMMs reveal that current models fall short of real-world application requirements. Even state-of-the-art LMMs exhibit substantial performance degradation across diverse tasks and scenarios. These findings reveal a significant gap between performance on current benchmarks and effectiveness in real-world applications. We release the full dataset and evaluation toolkit at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2605.03903
PDF
-
ATLAS: Article Tracking, Linking, and Analysis of Swedish Encyclopedias
2026-05-04 11:08:59Albin Andersson, Salam Jonasson, Fredrik Wastring, Pierre NuguesAbstract
The digitization of old encyclopedias represents an important step to improve access to historically structured knowledge. Often, however, this process does not go beyond an optical character recognition, leaving all the underlying structure unexploited. In addition, many encyclopedias had multiple editions reflecting the evolution of knowledge. The lack of structure in the raw text makes it difficult to track changes across these editions. In this work, we built a pipeline to restore the text structure, where we extract the headwords and identify entries; categorize the entities; match entries across editions; and link entries to a Wikidata item. We applied this pipeline to the four major editions of \textit{Nordisk familjebok}, an authoritative Swedish encyclopedia published between 1876 and 1951. We could extract the headwords with an F1 score of 97.8\% and we obtained an F1 score of 93.4\% on the headword classification. On a small-scale evaluation, we reached a 93\% precision on the cross-edition matching, 85\% precision and 16.5\% recall on the Wikidata linking. This shows that an automated approach to digitized historical knowledge is possible. This should facilitate the preservation of general knowledge and the understanding of knowledge transmission. The datasets and programs are available online.
Abstract (translated)
URL
https://arxiv.org/abs/2605.02466
PDF
-
When Good OCR Is Not Enough: Benchmarking OCR Robustness for Retrieval-Augmented Generation
2026-04-29 09:42:42Lin Sun, Wang Dexian, Jingang Huang, Linglin Zhang, Change Jia, Zhengwei Cheng, Xiangzheng ZhangAbstract
Industrial Retrieval-Augmented Generation (RAG) systems depend on optical character recognition (OCR) to transform visual documents into text. Existing OCR benchmarks rely on character-level metrics, which inadequately measure downstream RAG effectiveness under real-world conditions. We introduce an OCR benchmark for industrial RAG systems covering 11 challenging document types, including extreme layouts, high-resolution pages, complex or watermarked backgrounds, historical documents with non-standard reading orders, visually decorated text, and documents containing tables and mathematical formulas. Evaluating recent SOTA OCR models under a controlled OCR-first RAG pipeline shows clear performance degradation on realistic industrial documents despite strong conventional benchmark scores. We find that high OCR accuracy does not necessarily translate into strong downstream RAG performance: structural and semantic errors can cause substantial retrieval failures even when WER/CER remains low. Further analysis shows that this mismatch is category-dependent, arises through both retrieval-side and downstream generation-side failures, and remains stable across representative OCR-first pipeline choices. The benchmark is publicly available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2605.00911
PDF
-
Benchmarking OCR Pipelines with Adaptive Enhancement for Multi-Domain Retail Bill Digitization
2026-04-28 03:31:27Vijaysinh GaikwadAbstract
The digitization of multi-domain retail billing documents remains a challenging task due to variability in scan quality, layout heterogeneity, and domain diversity across commercial sectors. This paper proposes and benchmarks an intelligent, quality-aware adaptive Optical Character Recognition (OCR) pipeline for retail bill digitization spanning five domains: grocery stores, restaurants, hardware shops, footwear outlets, and clothing retailers. The proposed system integrates a Convolutional Neural Network (CNN)-based image enhancement module trained via self-supervised denoising, a Laplacian variance-based image quality analyzer with three-tier routing, a confidence-driven adaptive feedback loop with iterative retry, and an NLP-based post-OCR correction layer. Experiments were conducted on a real-world dataset of 360 heterogeneous retail bill images. Ground truth for quantitative evaluation was generated using an OCR ensemble majority voting strategy, a validated approach for scenarios without manual annotation. The proposed pipeline achieves a Character Error Rate (CER) of 18.4% and Word Error Rate (WER) of 27.6%, representing improvements of 26.4% and 31.2% respectively over the Raw Tesseract baseline. The pipeline additionally achieves a text density of 108.3 words per image, a noise ratio of 2.3%, and a processing time of 3.64 seconds per image - a 6.4x speed advantage over EasyOCR. Image quality PSNR analysis on enhanced MEDIUM and LOW quality images yields an average of 28.7 dB, confirming meaningful enhancement. These results establish a reproducible benchmark for multi-domain retail bill OCR research.
Abstract (translated)
URL
https://arxiv.org/abs/2604.25176
PDF
-
Reading in the Dark: Low-light Scene Text Recognition
2026-04-26 12:57:48Xuanshuo Fu, Lei Kang, Ernest Valveny, Dimosthenis Karatzas, Javier Vazquez-CorralAbstract
Accurate text recognition in low-light environments is essential for intelligent systems in applications ranging from autonomous vehicles to smart surveillance. However, challenges such as poor illumination and noise interference remain underexplored. To address this gap, we introduce LSTR, a large-scale Low-light Scene Text Recognition dataset comprising 11,273 low-light images generated from well-lit datasets (ICDAR2015, IIIT5K, and WordArt), along with ESTR, which includes 60 real nighttime street-scene images in English and Spanish for exclusive evaluation. We explore two solution strategies: (1) employing Optical Character Recognition (OCR) models with fine-tuning and LoRA-based fine-tuning and (2) a joint training strategy that integrates a low-light image enhancement (LLIE) module with an OCR model. In particular, we propose a novel re-render LLIE (RLLIE) module, which demonstrates improved performance on real-world data. Through extensive experimentation, we analyze various training strategies and address a key research question: \emph{How bright is bright enough for effective scene text recognition?} Our results indicate that standalone LLIE or OCR models perform inadequately under low-light conditions, highlighting the advantages of specialized, jointly trained text-centric approaches. Additionally, we provide a comprehensive benchmark to support future research in robust low-light scene text recognition. this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2604.23685
PDF
-
MESA: A Training-Free Multi-Exemplar Deep Framework for Restoring Ancient Inscription Textures
2026-04-19 11:38:03Vasileios Toulatzis, Ioannis FudosAbstract
Ancient inscriptions frequently suffer missing or corrupted regions from fragmentation, erosion, or other damage, hindering reading, and analysis. We review prior image restoration methods and their applicability to inscription image recovery, then introduce MESA (Multi-Exemplar, Style-Aware) -an image-level restoration method that uses well-preserved exemplar inscriptions (from the same epigraphic monument, material, or similar letterforms) to guide reconstruction of damaged text. MESA encodes VGG19 convolutional features as Gram matrices to capture exemplar texture, style, and stroke structure; for each neural network layer it selects the exemplar minimizing Mean-Squared Displacement (MSD) to the damaged input. Layer-wise contribution weights are derived from Optical Character Recognition-estimated character widths in the exemplar set to bias filters toward scales matching letter geometry, and a training mask preserves intact regions so synthesis is restricted to damaged areas. We also summarize prior network architectures and exemplar and single-image synthesis, inpainting, and Generative Adversarial Network (GAN) approaches, highlighting limitations that MESA addresses. Comparative experiments demonstrate the advantages of MESA. Finally, we provide a practical roadmap for choosing restoration strategies given available exemplars and metadata.
Abstract (translated)
URL
https://arxiv.org/abs/2604.17390
PDF
-
Aligning What Vision-Language Models See and Perceive with Adaptive Information Flow
2026-04-17 08:07:22Chengxin Liu, Wonseok Choi, Chenshuang Zhang, Tae-Hyun OhAbstract
Vision-Language Models (VLMs) have demonstrated strong capability in a wide range of tasks such as visual recognition, document parsing, and visual grounding. Nevertheless, recent work shows that while VLMs often manage to capture the correct image region corresponding to the question, they do not necessarily produce the correct answers. In this work, we demonstrate that this misalignment could be attributed to suboptimal information flow within VLMs, where text tokens distribute too much attention to irrelevant visual tokens, leading to incorrect answers. Based on the observation, we show that modulating the information flow during inference can improve the perception capability of VLMs. The idea is that text tokens should only be associated with important visual tokens during decoding, eliminating the interference of irrelevant regions. To achieve this, we propose a token dynamics-based method to determine the importance of visual tokens, where visual tokens that exhibit distinct activation patterns during different decoding stages are viewed as important. We apply our approach to representative open-source VLMs and evaluate on various datasets, including visual question answering, visual grounding and counting, optical character recognition, and object hallucination. The results show that our approach significantly improves the performance of baselines. Project page: this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2604.15809
PDF
-
GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts
2026-04-14 17:12:41Amir Hossein Kargaran, Nafiseh Nikeghbal, Jana Diesner, Fran\c{c}ois Yvon, Hinrich Sch\"utzeAbstract
Optical character recognition (OCR) has advanced rapidly with the rise of vision-language models, yet evaluation has remained concentrated on a small cluster of high- and mid-resource scripts. We introduce GlotOCR Bench, a comprehensive benchmark evaluating OCR generalization across 100+ Unicode scripts. Our benchmark comprises clean and degraded image variants rendered from real multilingual texts. Images are rendered using fonts from the Google Fonts repository, shaped with HarfBuzz and rasterized with FreeType, supporting both LTR and RTL scripts. Samples of rendered images were manually reviewed to verify correct rendering across all scripts. We evaluate a broad suite of open-weight and proprietary vision-language models and find that most perform well on fewer than ten scripts, and even the strongest frontier models fail to generalize beyond thirty scripts. Performance broadly tracks script-level pretraining coverage, suggesting that current OCR systems rely on language model pretraining as much as on visual recognition. Models confronted with unfamiliar scripts either produce random noise or hallucinate characters from similar scripts they already know. We release the benchmark and pipeline for reproducibility. Pipeline Code: this https URL, Benchmark: this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2604.12978
PDF
-
Dual-Control Frequency-Aware Diffusion Model for Depth-Dependent Optical Microrobot Microscopy Image Generation
2026-04-13 16:24:08Lan Wei, Zongcai Tan, Kangyi Lu, Jian-Qing Zheng, Dandan ZhangAbstract
Optical microrobots actuated by optical tweezers (OT) are important for cell manipulation and microscale assembly, but their autonomous operation depends on accurate 3D perception. Developing such perception systems is challenging because large-scale, high-quality microscopy datasets are scarce, owing to complex fabrication processes and labor-intensive annotation. Although generative AI offers a promising route for data augmentation, existing generative adversarial network (GAN)-based methods struggle to reproduce key optical characteristics, particularly depth-dependent diffraction and defocus effects. To address this limitation, we propose Du-FreqNet, a dual-control, frequency-aware diffusion model for physically consistent microscopy image synthesis. The framework features two independent ControlNet branches to encode microrobot 3D point clouds and depth-specific mesh layers, respectively. We introduce an adaptive frequency-domain loss that dynamically reweights high- and low-frequency components based on the distance to the focal plane. By leveraging differentiable FFT-based supervision, Du-FreqNet captures physically meaningful frequency distributions often missed by pixel-space methods. Trained on a limited dataset (e.g., 80 images per pose), our model achieves controllable, depth-dependent image synthesis, improving SSIM by 20.7% over baselines. Extensive experiments demonstrate that Du-FreqNet generalizes effectively to unseen poses and significantly enhances downstream tasks, including 3D pose and depth estimation, thereby facilitating robust closed-loop control in microrobotic systems.
Abstract (translated)
URL
https://arxiv.org/abs/2604.11680
PDF
-
DocRevive: A Unified Pipeline for Document Text Restoration
2026-04-11 07:50:20Kunal Purkayastha, Ayan Banerjee, Josep Llados, Umapada PalAbstract
In Document Understanding, the challenge of reconstructing damaged, occluded, or incomplete text remains a critical yet unexplored problem. Subsequent document understanding tasks can benefit from a document reconstruction process. In response, this paper presents a novel unified pipeline combining state-of-the-art Optical Character Recognition (OCR), advanced image analysis, masked language modeling, and diffusion-based models to restore and reconstruct text while preserving visual integrity. We create a synthetic dataset of 30{,}078 degraded document images that simulates diverse document degradation scenarios, setting a benchmark for restoration tasks. Our pipeline detects and recognizes text, identifies degradation with an occlusion detector, and uses an inpainting model for semantically coherent reconstruction. A diffusion-based module seamlessly reintegrates text, matching font, size, and alignment. To evaluate restoration quality, we propose a Unified Context Similarity Metric (UCSM), incorporating edit, semantic, and length similarities with a contextual predictability measure that penalizes deviations when the correct text is contextually obvious. Our work advances document restoration, benefiting archival research and digital preservation while setting a new standard for text reconstruction. The OPRB dataset and code are available at \href{this https URL}{Hugging Face} and \href{this https URL}{Github} respectively.
Abstract (translated)
URL
https://arxiv.org/abs/2604.10077
PDF
-
AtlasOCR: Building the First Open-Source Darija OCR Model with Vision Language Models
2026-04-09 10:38:23Imane Momayiz, Soufiane Ait Elaouad, Abdeljalil Elmajjodi, Haitame BouananeAbstract
Darija, the Moroccan Arabic dialect, is rich in visual content yet lacks specialized Optical Character Recognition (OCR) tools. This paper introduces AtlasOCR, the first open-source Darija OCR model built by fine-tuning a 3B parameter Vision Language Model (VLM). We detail our comprehensive approach, from curating a unique Darija-specific dataset leveraging both synthetic generation with our OCRSmith library and carefully sourced real-world data, to implementing efficient fine-tuning strategies. We utilize QLoRA and Unsloth for parameter-efficient training of Qwen2.5-VL 3B and present comprehensive ablation studies optimizing key hyperparameters. Our evaluation on the newly curated AtlasOCRBench and the established KITAB-Bench demonstrates state-of-the-art performance, challenging larger models and highlighting AtlasOCR's robustness and generalization capabilities for both Darija and standard Arabic OCR tasks.
Abstract (translated)
URL
https://arxiv.org/abs/2604.08070
PDF
-
Multi-Head Attention based interaction-aware architecture for Bangla Handwritten Character Recognition: Introducing a Primary Dataset
2026-04-08 13:18:08Mirza Raquib, Asif Pervez Polok, Kedar Nath Biswas, Farida Siddiqi Prity, Saydul Akbar Murad, Nick RahimiAbstract
Character recognition is the fundamental part of an optical character recognition (OCR) system. Word recognition, sentence transcription, document digitization, and language processing are some of the higher-order activities that can be done accurately through character recognition. Nonetheless, recognizing handwritten Bangla characters is not an easy task because they are written in different styles with inconsistent stroke patterns and a high degree of visual character resemblance. The datasets available are usually limited in intra-class and inequitable in class distribution. We have constructed a new balanced dataset of Bangla written characters to overcome those problems. This consists of 78 classes and each class has approximately 650 samples. It contains the basic characters, composite (Juktobarno) characters and numerals. The samples were a diverse group comprising a large age range and socioeconomic groups. Elementary and high school students, university students, and professionals are the contributing factors. The sample also has right and left-handed writers. We have further proposed an interaction-aware hybrid deep learning architecture that integrates EfficientNetB3, Vision Transformer, and Conformer modules in parallel. A multi-head cross-attention fusion mechanism enables effective feature interaction across these components. The proposed model achieves 98.84% accuracy on the constructed dataset and 96.49% on the external CHBCR benchmark, demonstrating strong generalization capability. Grad-CAM visualizations further provide interpretability by highlighting discriminative regions. The dataset and source code of this research is publicly available at: this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2604.09717
PDF
-
LLM-based Schema-Guided Extraction and Validation of Missing-Person Intelligence from Heterogeneous Data Sources
2026-04-08 01:35:56Joshua Castillo, Ravi MukkamalaAbstract
Missing-person and child-safety investigations rely on heterogeneous case documents, including structured forms, bulletin-style posters, and narrative web profiles. Variations in layout, terminology, and data quality impede rapid triage, large-scale analysis, and search-planning workflows. This paper introduces the Guardian Parser Pack, an AI-driven parsing and normalization pipeline that transforms multi-source investigative documents into a unified, schema-compliant representation suitable for operational review and downstream spatial modeling. The proposed system integrates (i) multi-engine PDF text extraction with Optical Character Recognition (OCR) fallback, (ii) rule-based source identification with source-specific parsers, (iii) schema-first harmonization and validation, and (iv) an optional Large Language Model (LLM)-assisted extraction pathway incorporating validator-guided repair and shared geocoding services. We present the system architecture, key implementation decisions, and output design, and evaluate performance using both gold-aligned extraction metrics and corpus-level operational indicators. On a manually aligned subset of 75 cases, the LLM-assisted pathway achieved substantially higher extraction quality than the deterministic comparator (F1 = 0.8664 vs. 0.2578), while across 517 parsed records per pathway it also improved aggregate key-field completeness (96.97\% vs. 93.23\%). The deterministic pathway remained much faster (mean runtime 0.03 s/record vs. 3.95 s/record for the LLM pathway). In the evaluated run, all LLM outputs passed initial schema validation, so validator-guided repair functioned as a built-in safeguard rather than a contributor to the observed gains. These results support controlled use of probabilistic AI within a schema-first, auditable pipeline for high-stakes investigative settings.
Abstract (translated)
URL
https://arxiv.org/abs/2604.06571
PDF
-
The Character Error Vector: Decomposable errors for page-level OCR evaluation
2026-04-07 17:56:06Jonathan Bourne, Mwiza Simbeye, Joseph NockelsAbstract
The Character Error Rate (CER) is a key metric for evaluating the quality of Optical Character Recognition (OCR). However, this metric assumes that text has been perfectly parsed, which is often not the case. Under page-parsing errors, CER becomes undefined, limiting its use as a metric and making evaluating page-level OCR challenging, particularly when using data that do not share a labelling schema. We introduce the Character Error Vector (CEV), a bag-of-characters evaluator for OCR. The CEV can be decomposed into parsing and OCR, and interaction error components. This decomposability allows practitioners to focus on the part of the Document Understanding pipeline that will have the greatest impact on overall text extraction quality. The CEV can be implemented using a variety of methods, of which we demonstrate SpACER (Spatially Aware Character Error Rate) and a Character distribution method using the Jensen-Shannon Distance. We validate the CEV's performance against other metrics: first, the relationship with CER; then, parse quality; and finally, as a direct measure of page-level OCR quality. The validation process shows that the CEV is a valuable bridge between parsing metrics and local metrics like CER. We analyse a dataset of archival newspapers made of degraded images with complex layouts and find that state-of-the-art end-to-end models are outperformed by more traditional pipeline approaches. Whilst the CEV requires character-level positioning for optimal triage, thresholding on easily available values can predict the main error source with an F1 of 0.91. We provide the CEV as part of a Python library to support Document understanding research.
Abstract (translated)
字符错误率（CER）是评估光学字符识别（OCR）质量的关键指标。然而，该指标假设文本已被完美解析，而实际情况往往并非如此。在存在页面解析错误时，CER变得无法定义，这限制了其作为评估指标的应用，并使得对页面级OCR的评估变得困难，尤其是在使用标注模式不一致的数据时。我们提出了字符错误向量（CEV），一种用于OCR的字符袋评估器。CEV可分解为解析错误、OCR错误及交互误差三部分。这种可分解性使从业者能够聚焦于文档理解流水线中对整体文本提取质量影响最大的环节。CEV可通过多种方法实现，其中我们展示了空间感知字符错误率（SpACER）以及基于Jensen-Shannon距离的字符分布方法。我们通过以下方式验证CEV相对于其他指标的性能：首先，与CER的关系；其次，解析质量；最后，作为页面级OCR质量的直接度量。验证过程表明，CEV在解析指标与CER等局部指标之间建立了有价值的桥梁。我们分析了一个由布局复杂、图像退化的档案报纸组成的数据集，发现最先进的端到端模型表现不如更传统的流水线方法。尽管CEV需要字符级定位以实现最佳分类，但基于易获取的数值进行阈值设定即可以0.91的F1值预测主要误差来源。我们已将CEV作为Python库的一部分提供，以支持文档理解研究。
URL
https://arxiv.org/abs/2604.06160
PDF
-
Toward Unified Fine-Grained Vehicle Classification and Automatic License Plate Recognition
2026-04-07 00:10:56Gabriel E. Lima, Valfride Nascimento, Eduardo Santos, Eduil Nascimento Jr, Rayson Laroca, David MenottiAbstract
Extracting vehicle information from surveillance images is essential for intelligent transportation systems, enabling applications such as traffic monitoring and criminal investigations. While Automatic License Plate Recognition (ALPR) is widely used, Fine-Grained Vehicle Classification (FGVC) offers a complementary approach by identifying vehicles based on attributes such as color, make, model, and type. Although there have been advances in this field, existing studies often assume well-controlled conditions, explore limited attributes, and overlook FGVC integration with ALPR. To address these gaps, we introduce UFPR-VeSV, a dataset comprising 24,945 images of 16,297 unique vehicles with annotations for 13 colors, 26 makes, 136 models, and 14 types. Collected from the Military Police of Paraná (Brazil) surveillance system, the dataset captures diverse real-world conditions, including partial occlusions, nighttime infrared imaging, and varying lighting. All FGVC annotations were validated using license plate information, with text and corner annotations also being provided. A qualitative and quantitative comparison with established datasets confirmed the challenging nature of our dataset. A benchmark using five deep learning models further validated this, revealing specific challenges such as handling multicolored vehicles, infrared images, and distinguishing between vehicle models that share a common platform. Additionally, we apply two optical character recognition models to license plate recognition and explore the joint use of FGVC and ALPR. The results highlight the potential of integrating these complementary tasks for real-world applications. The UFPR-VeSV dataset is publicly available at: this https URL.
Abstract (translated)
从监控图像中提取车辆信息对于智能交通系统至关重要，可支持交通监控和刑事调查等应用。虽然自动车牌识别（ALPR）已广泛应用，但细粒度车辆分类（FGVC）通过识别颜色、品牌、型号和类型等属性提供了互补方法。尽管该领域已有进展，现有研究常假设条件受控、探索的属性有限，且忽视了FGVC与ALPR的整合。为弥补这些不足，我们推出UFPR-VeSV数据集，该数据集包含16,297辆独特车辆的24,945张图像，标注了13种颜色、26个品牌、136种型号和14种类型。数据集源自巴西巴拉那州军事警察监控系统，涵盖了多种真实世界场景，包括部分遮挡、夜间红外成像和光照变化。所有FGVC标注均通过车牌信息验证，同时提供了文本和角点标注。与现有数据集的定性和定量比较证实了本数据集的挑战性。使用五种深度学习模型的基准测试进一步验证了这一点，揭示了处理多色车辆、红外图像以及区分同平台车型等具体挑战。此外，我们应用两种光学字符识别模型进行车牌识别，并探索了FGVC与ALPR的联合使用。结果凸显了整合这些互补任务在现实应用中的潜力。UFPR-VeSV数据集已公开提供，地址为：此https链接。
URL
https://arxiv.org/abs/2604.05271
PDF
-
A ROS 2 Wrapper for Florence-2: Multi-Mode Local Vision-Language Inference for Robotic Systems
2026-04-01 17:29:59J. E. Dom\'inguez-VidalAbstract
Foundation vision-language models are becoming increasingly relevant to robotics because they can provide richer semantic perception than narrow task-specific pipelines. However, their practical adoption in robot software stacks still depends on reproducible middleware integrations rather than on model quality alone. Florence-2 is especially attractive in this regard because it unifies captioning, optical character recognition, open-vocabulary detection, grounding and related vision-language tasks within a comparatively manageable model size. This article presents a ROS 2 wrapper for Florence-2 that exposes the model through three complementary interaction modes: continuous topic-driven processing, synchronous service calls and asynchronous actions. The wrapper is designed for local execution and supports both native installation and Docker container deployment. It also combines generic JSON outputs with standard ROS 2 message bindings for detection-oriented tasks. A functional validation is reported together with a throughput study on several GPUs, showing that local deployment is feasible with consumer grade hardware. The repository is publicly available here: this https URL
Abstract (translated)
基础视觉语言模型正日益与机器人技术相关，因为它们能比狭窄的特定任务流水线提供更丰富的语义感知。然而，这些模型在机器人软件栈中的实际应用仍取决于可复现的中间件集成，而不仅仅是模型质量本身。在这方面，Florence-2 尤为突出，因为它以相对可控的模型规模统一了字幕生成、光学字符识别、开放词汇检测、定位及相关视觉语言任务。本文介绍了 Florence-2 的 ROS 2 封装器，该封装器通过三种互补的交互模式提供模型访问：连续主题驱动处理、同步服务调用和异步动作。该封装器设计用于本地执行，支持原生安装和 Docker 容器部署。它还将通用 JSON 输出与面向检测任务的标准 ROS 2 消息绑定相结合。文章报告了功能验证及在多个 GPU 上的吞吐量研究，表明在消费级硬件上实现本地部署是可行的。代码仓库已公开，链接如下：此 https URL
URL
https://arxiv.org/abs/2604.01179
PDF
-
When VLMs 'Fix' Students: Identifying and Penalizing Over-Correction in the Evaluation of Multi-line Handwritten Math OCR
2026-04-01 05:27:52Jin Seong, Wencke Liermann, Minho Kim, Jong-hun Shin, Soojong LimAbstract
Accurate transcription of handwritten mathematics is crucial for educational AI systems, yet current benchmarks fail to evaluate this capability properly. Most prior studies focus on single-line expressions and rely on lexical metrics such as BLEU, which fail to assess the semantic reasoning across multi-line student solutions. In this paper, we present the first systematic study of multi-line handwritten math Optical Character Recognition (OCR), revealing a critical failure mode of Vision-Language Models (VLMs): over-correction. Instead of faithfully transcribing a student's work, these models often "fix" errors, thereby hiding the very mistakes an educational assessment aims to detect. To address this, we propose PINK (Penalized INK-based score), a semantic evaluation metric that leverages a Large Language Model (LLM) for rubric-based grading and explicitly penalizes over-correction. Our comprehensive evaluation of 15 state-of-the-art VLMs on the FERMAT dataset reveals substantial ranking reversals compared to BLEU: models like GPT-4o are heavily penalized for aggressive over-correction, whereas Gemini 2.5 Flash emerges as the most faithful transcriber. Furthermore, human expert studies show that PINK aligns significantly better with human judgment (55.0% preference over BLEU's 39.5%), providing a more reliable evaluation framework for handwritten math OCR in educational settings.
Abstract (translated)
URL
https://arxiv.org/abs/2604.22774
PDF
-
Q-Mask: Query-driven Causal Masks for Text Anchoring in OCR-Oriented Vision-Language Models
2026-03-31 19:09:55Longwei Xu, Feng Feng, Shaojie Zhang, Xin Chen, Hang Li, Anan Du, Hailong Yu, Pei Fu, Zhenbo Luo, Jian LuanAbstract
Optical Character Recognition (OCR) is increasingly regarded as a foundational capability for modern vision-language models (VLMs), enabling them not only to read text in images but also to support downstream reasoning in real-world visual question answering (VQA). However, practical applications further require reliable text anchors, i.e., accurately grounding queried text to its corresponding spatial region. To systematically evaluate this capability, we introduce TextAnchor-Bench (TABench), a benchmark for fine-grained text-region grounding, which reveals that both general-purpose and OCR-specific VLMs still struggle to establish accurate and stable text anchors. To address this limitation, we propose Q-Mask, a precise OCR framework built upon a causal query-driven mask decoder (CQMD). Inspired by chain-of-thought reasoning, Q-Mask performs causal visual decoding that sequentially generates query-conditioned visual masks before producing the final OCR output. This visual CoT paradigm disentangles where the text is from what the text is, enforcing grounded evidence acquisition prior to recognition and enabling explicit text anchor construction during inference. To train CQMD, we construct TextAnchor-26M, a large-scale dataset of image-text pairs annotated with fine-grained masks corresponding to specific textual elements, encouraging stable text-region correspondences and injecting strong spatial priors into VLM training. Extensive experiments demonstrate that Q-Mask substantially improves text anchoring and understanding across diverse visual scenes.
Abstract (translated)
光学字符识别（OCR）正日益被视为现代视觉语言模型（VLM）的基础能力，它不仅使模型能够读取图像中的文本，还能支持现实世界视觉问答（VQA）中的下游推理。然而，实际应用进一步要求可靠的文本锚点，即准确地将查询文本定位到其对应的空间区域。为系统评估这一能力，我们推出了TextAnchor-Bench（TABench），一个用于细粒度文本-区域定位的基准测试，该测试揭示通用型和OCR专用型VLM仍难以建立准确稳定的文本锚点。为应对这一局限，我们提出了Q-Mask，一个基于因果查询驱动掩码解码器（CQMD）构建的精确OCR框架。受思维链推理启发，Q-Mask执行因果视觉解码，在生成最终OCR输出前，依次生成查询条件化的视觉掩码。这种视觉思维链范式将“文本位置”与“文本内容”解耦，强制在识别前获取有根据的证据，并在推理过程中实现显式的文本锚点构建。为训练CQMD，我们构建了TextAnchor-26M，一个大规模图像-文本对数据集，其中标注了与特定文本元素对应的细粒度掩码，以促进稳定的文本-区域对应关系，并将强空间先验注入VLM训练。大量实验表明，Q-Mask在多样化的视觉场景中显著提升了文本锚定与理解能力。
URL
https://arxiv.org/abs/2604.00161
PDF
-
Conditional Polarization Guidance for Camouflaged Object Detection
2026-03-31 17:03:06QIfan Zhang, Hao Wang, Xiangrong Qin, Ruijie LiAbstract
Camouflaged object detection (COD) aims to identify targets that are highly blended with their backgrounds. Recent works have shown that the optical characteristics of polarization cues play a significant role in improving camouflaged object detection. However, most existing polarization-based approaches depend on complex visual encoders and fusion mechanisms, leading to increased model complexity and computational overhead, while failing to fully explore how polarization can explicitly guide hierarchical RGB representation learning. To address these limitations, we propose CPGNet, an asymmetric RGB-polarization framework that introduces a conditional polarization guidance mechanism to explicitly regulate RGB feature learning for camouflaged object detection. Specifically, we design a lightweight polarization interaction module that jointly models these complementary cues and generates reliable polarization guidance in a unified manner. Unlike conventional feature fusion strategies, the proposed conditional guidance mechanism dynamically modulates RGB features using polarization priors, enabling the network to focus on subtle discrepancies between camouflaged objects and their backgrounds. Furthermore, we introduce a polarization edge-guided frequency refinement strategy that enhances high-frequency components under polarization constraints, effectively breaking camouflage patterns. Finally, we develop an iterative feedback decoder to perform coarse-to-fine feature calibration and progressively refine camouflage prediction. Extensive experiments on polarization datasets across multiple tasks, along with evaluations on non-polarization datasets, demonstrate that CPGNet consistently outperforms state-of-the-art methods.
Abstract (translated)
伪装目标检测（COD）旨在识别与背景高度融合的目标。近期研究表明，偏振线索的光学特性对提升伪装目标检测效果具有重要作用。然而，现有基于偏振的方法大多依赖复杂的视觉编码器和融合机制，导致模型复杂度与计算开销增加，且未能充分探索偏振如何显式指导分层RGB表征学习。针对这些局限，我们提出CPGNet——一种RGB-偏振非对称框架，通过引入条件偏振引导机制显式调控RGB特征学习以优化伪装目标检测。具体而言，我们设计了轻量级偏振交互模块，统一建模互补线索并生成可靠的偏振引导信号。与传统特征融合策略不同，该条件引导机制利用偏振先验动态调制RGB特征，使网络聚焦伪装目标与背景间的细微差异。此外，我们提出偏振边缘引导的频率细化策略，在偏振约束下增强高频分量以有效打破伪装模式。最后，我们开发了迭代反馈解码器，通过粗到细的特征校准逐步优化伪装预测。在跨任务偏振数据集上的大量实验及非偏振数据集评估表明，CPGNet持续优于最先进方法。
URL
https://arxiv.org/abs/2603.30008
PDF
-
L-ReLF: A Framework for Lexical Dataset Creation
2026-03-31 07:19:00Anass Sedrati, Mounir Afifi, Reda BenkhadraAbstract
This paper introduces the L-ReLF (Low-Resource Lexical Framework), a novel, reproducible methodology for creating high-quality, structured lexical datasets for underserved languages. The lack of standardized terminology, exemplified by Moroccan Darija, poses a critical barrier to knowledge equity in platforms like Wikipedia, often forcing editors to rely on inconsistent, ad-hoc methods to create new words in their language. Our research details the technical pipeline developed to overcome these challenges. We systematically address the difficulties of working with low-resource data, including source identification, utilizing Optical Character Recognition (OCR) despite its bias towards Modern Standard Arabic, and rigorous post-processing to correct errors and standardize the data model. The resulting structured dataset is fully compatible with Wikidata Lexemes, serving as a vital technical resource. The L-ReLF methodology is designed for generalizability, offering other language communities a clear path to build foundational lexical data for downstream NLP applications, such as Machine Translation and morphological analysis.
Abstract (translated)
本文介绍了L-ReLF（低资源词汇框架），这是一种新颖且可复现的方法论，旨在为资源匮乏的语言创建高质量、结构化的词汇数据集。以摩洛哥达里贾语为例，缺乏标准化术语的现象严重阻碍了维基百科等平台上的知识公平性，往往迫使编辑者依赖不一致的临时方法来为本语言创造新词。本研究详细阐述了为应对这些挑战而开发的技术流程。我们系统性地解决了处理低资源数据的困难，包括来源识别、在光学字符识别（OCR）对现代标准阿拉伯语存在偏见的情况下仍加以利用，以及通过严格的后处理来纠正错误并标准化数据模型。最终生成的结构化数据集与Wikidata Lexemes完全兼容，作为一项重要的技术资源。L-ReLF方法论设计具有通用性，为其他语言社区构建面向下游自然语言处理应用（如机器翻译和形态分析）的基础词汇数据提供了清晰路径。
URL
https://arxiv.org/abs/2603.29346
PDF
