Abstract
Handwritten Mathematical Expression Recognition (HMER) remains a persistent challenge in Optical Character Recognition (OCR) due to the inherent freedom of symbol layout and variability in handwriting styles. Prior methods have faced performance bottlenecks, proposing isolated architectural modifications that are difficult to integrate coherently into a unified framework. Meanwhile, recent advances in pretrained vision-language models (VLMs) have demonstrated strong cross-task generalization, offering a promising foundation for developing unified solutions. In this paper, we introduce Uni-MuMER, which fully fine-tunes a VLM for the HMER task without modifying its architecture, effectively injecting domain-specific knowledge into a generalist framework. Our method integrates three data-driven tasks: Tree-Aware Chain-of-Thought (Tree-CoT) for structured spatial reasoning, Error-Driven Learning (EDL) for reducing confusion among visually similar characters, and Symbol Counting (SC) for improving recognition consistency in long expressions. Experiments on the CROHME and HME100K datasets show that Uni-MuMER achieves new state-of-the-art performance, surpassing the best lightweight specialized model SSAN by 16.31% and the top-performing VLM Gemini2.5-flash by 24.42% in the zero-shot setting. Our datasets, models, and code are open-sourced at: this https URL
Abstract (translated)
手写数学表达式识别（HMER）在光学字符识别（OCR）中仍然是一项持续的挑战，这是由于符号布局的自由度和书写风格的变化所致。先前的方法由于提出孤立的架构修改而遇到了性能瓶颈，这些修改难以整合到一个统一框架内。与此同时，近期预训练视觉语言模型（VLMs）在跨任务泛化方面表现出色，为开发统一体解决方案提供了有前景的基础。本文中，我们介绍了一种名为Uni-MuMER的方法，该方法无需修改其架构即可对VLM进行全面微调，从而有效地将领域特定知识注入到通用框架中。我们的方法整合了三项数据驱动的任务：结构化空间推理的树感知链式思维（Tree-CoT）、减少视觉相似字符混淆的错误驱动学习（EDL）以及提高长表达式识别一致性的符号计数（SC）。在CROHME和HME100K数据集上的实验表明，Uni-MuMER达到了新的最先进的性能，在零样本设置下，其表现优于最轻量级的专业模型SSAN 16.31%，优于顶级的VLM Gemini2.5-flash 24.42%。 我们的数据集、模型和代码已开源：[请在此处插入链接]（请注意原文中的具体网址无法直接显示，请访问原论文获取准确的URL）。
URL
https://arxiv.org/abs/2505.23566
