Abstract
Large Multimodal Models (LMMs) have become increasingly versatile, accompanied by impressive Optical Character Recognition (OCR) related capabilities. Existing OCR-related benchmarks emphasize evaluating LMMs' abilities of relatively simple visual question answering, visual-text parsing, etc. However, the extent to which LMMs can deal with complex logical reasoning problems based on OCR cues is relatively unexplored. To this end, we introduce the Reasoning-OCR benchmark, which challenges LMMs to solve complex reasoning problems based on the cues that can be extracted from rich visual-text. Reasoning-OCR covers six visual scenarios and encompasses 150 meticulously designed questions categorized into six reasoning challenges. Additionally, Reasoning-OCR minimizes the impact of field-specialized knowledge. Our evaluation offers some insights for proprietary and open-source LMMs in different reasoning challenges, underscoring the urgent to improve the reasoning performance. We hope Reasoning-OCR can inspire and facilitate future research on enhancing complex reasoning ability based on OCR cues. Reasoning-OCR is publicly available at this https URL.
Abstract (translated)
大型多模态模型（LMMs）已经变得越来越灵活，并且伴随有令人印象深刻的光学字符识别（OCR）相关能力。现有的OCR相关的基准测试主要侧重于评估LMM在相对简单的视觉问答、图文解析等方面的能力，然而，LMM基于OCR线索处理复杂逻辑推理问题的程度则较少被探索。为此，我们引入了Reasoning-OCR基准测试，该测试挑战LMM根据从丰富图文中提取的线索解决复杂的推理问题。Reasoning-OCR涵盖了六种视觉场景，并包含了150个精心设计的问题，这些问题分为六个推理挑战类别。此外，Reasoning-OCR尽可能减少了领域专业知识的影响。我们的评估为专有和开源的LMM在不同推理挑战中的表现提供了一些见解，强调了提升推理性能的紧迫性。我们希望Reasoning-OCR能激发并促进未来基于OCR线索增强复杂推理能力的研究。Reasoning-OCR可在[此处](https://this https URL)公开获取。 以下是更准确的翻译： 大型多模态模型（LMMs）已变得越来越灵活，并且伴随有令人印象深刻的光学字符识别（OCR）相关能力。现有的OCR相关的基准测试主要侧重于评估LMM在相对简单的视觉问答、图文解析等方面的能力，但基于OCR线索处理复杂逻辑推理问题的程度则较少被探索。为此，我们引入了Reasoning-OCR基准测试，该测试挑战LMM根据从丰富图文中提取的线索解决复杂的推理问题。Reasoning-OCR涵盖了六种视觉场景，并包含了150个精心设计的问题，这些问题是按照六个推理挑战进行分类的。此外，Reasoning-OCR尽可能减少了领域专业知识的影响。我们的评估为专有和开源的LMM在不同推理挑战中的表现提供了一些见解，强调了提升推理性能的紧迫性。我们希望Reasoning-OCR能激发并促进未来基于OCR线索增强复杂推理能力的研究。Reasoning-OCR可在此[链接](https://this https URL)公开获取。
URL
https://arxiv.org/abs/2505.12766
