Abstract
Document intelligence requires accurate text extraction and reliable reasoning over document content. We introduce \textbf{DISCO}, a \emph{Document Intelligence Suite for COmparative Evaluation}, that evaluates optical character recognition (OCR) pipelines and vision-language models (VLMs) separately on parsing and question answering across diverse document types, including handwritten text, multilingual scripts, medical forms, infographics, and multi-page documents. Our evaluation shows that performance varies substantially across tasks and document characteristics, underscoring the need for complexity-aware approach selection. OCR pipelines are generally more reliable for handwriting and for long or multi-page documents, where explicit text grounding supports text-heavy reasoning, while VLMs perform better on multilingual text and visually rich layouts. Task-aware prompting yields mixed effects, improving performance on some document types while degrading it on others. These findings provide empirical guidance for selecting document processing strategies based on document structure and reasoning demands.
Abstract (translated)
文档智能需要精确的文本提取和对文档内容的可靠推理。我们推出了\textbf{DISCO}（\emph{面向对比评估的文档智能套件}），该工具分别评估光学字符识别（OCR）流水线和视觉-语言模型（VLMs）在多样化文档类型上的解析与问答能力，涵盖手写文本、多语言文本、医疗表格、信息图表及多页文档。评估结果显示，性能因任务和文档特征差异显著，凸显了选择复杂度感知方法的必要性。OCR流水线在手写内容及长文档或多页文档中通常更可靠，其中显式文本锚定支持文本密集型推理；而VLMs在多语言文本和视觉丰富布局上表现更优。任务感知提示产生混合效果：虽提升部分文档类型性能，却在另一些类型中导致下降。这些发现为基于文档结构与推理需求选择文档处理策略提供了实证指导。
URL
https://arxiv.org/abs/2603.23511
