Abstract
Visually Rich Documents (VRDs) are essential in academia, finance, medical fields, and marketing due to their multimodal information content. Traditional methods for extracting information from VRDs depend on expert knowledge and manual labor, making them costly and inefficient. The advent of deep learning has revolutionized this process, introducing models that leverage multimodal information vision, text, and layout along with pretraining tasks to develop comprehensive document representations. These models have achieved state-of-the-art performance across various downstream tasks, significantly enhancing the efficiency and accuracy of information extraction from VRDs. In response to the growing demands and rapid developments in Visually Rich Document Understanding (VRDU), this paper provides a comprehensive review of deep learning-based VRDU frameworks. We systematically survey and analyze existing methods and benchmark datasets, categorizing them based on adopted strategies and downstream tasks. Furthermore, we compare different techniques used in VRDU models, focusing on feature representation and fusion, model architecture, and pretraining methods, while highlighting their strengths, limitations, and appropriate scenarios. Finally, we identify emerging trends and challenges in VRDU, offering insights into future research directions and practical applications. This survey aims to provide a thorough understanding of VRDU advancements, benefiting both academic and industrial sectors.
Abstract (translated)
视觉丰富文档（VRDs）在学术界、金融界和医疗领域以及市场营销中具有多模态信息内容，因此在这些领域中是必不可少的。从VRDs中提取信息的传统方法依赖于专家知识和手动劳动，导致它们既耗时又效率低下。深度学习的出现彻底颠覆了这一过程，引入了利用多模态信息视觉、文本和布局以及预训练任务来构建全面文档表示的模型。这些模型在各种下游任务上都取得了最先进的性能，显著提高了从VRDs中提取信息的效率和准确性。 为了满足VRDU不断增长的需求和快速发展的趋势，本文对基于深度学习的VRDU框架进行了全面的回顾。我们系统地调查和分析了现有方法，并根据采用的策略和下游任务将它们分类。此外，我们比较了VRDU模型中使用的不同技术，重点关注特征表示和融合、模型架构和预训练方法，并强调了它们的优缺点以及适用的场景。最后，我们确定了VRDU中的新兴趋势和挑战，并为未来的研究方向和实际应用提供了见解。 本次调查旨在为VRDU的进展提供深入的理解，为学术和工业领域带来实际的益处。
URL
https://arxiv.org/abs/2408.01287
