Abstract
We present a new table structure recognition (TSR) approach, called TSRFormer, to robustly recognizing the structures of complex tables with geometrical distortions from various table images. Unlike previous methods, we formulate table separation line prediction as a line regression problem instead of an image segmentation problem and propose a new two-stage dynamic queries enhanced DETR based separation line regression approach, named DQ-DETR, to predict separation lines from table images directly. Compared to Vallina DETR, we propose three improvements in DQ-DETR to make the two-stage DETR framework work efficiently and effectively for the separation line prediction task: 1) A new query design, named Dynamic Query, to decouple single line query into separable point queries which could intuitively improve the localization accuracy for regression tasks; 2) A dynamic queries based progressive line regression approach to progressively regressing points on the line which further enhances localization accuracy for distorted tables; 3) A prior-enhanced matching strategy to solve the slow convergence issue of DETR. After separation line prediction, a simple relation network based cell merging module is used to recover spanning cells. With these new techniques, our TSRFormer achieves state-of-the-art performance on several benchmark datasets, including SciTSR, PubTabNet, WTW and FinTabNet. Furthermore, we have validated the robustness and high localization accuracy of our approach to tables with complex structures, borderless cells, large blank spaces, empty or spanning cells as well as distorted or even curved shapes on a more challenging real-world in-house dataset.
Abstract (translated)
我们提出了一种新的表格结构识别方法,称为TSR Former,以 robustly 识别从各种表格图像中产生的复杂的表格结构。与以前的方法不同,我们将表格分离线预测视为一条线回归问题,而不是图像分割问题,并提出了一种新的基于动态查询增强的分离线回归方法,称为DQ-DETR,直接从表格图像中预测分离线。与 VallinaDETR 相比,我们提出了 three 项改进,以使两阶段 DETR 框架在分离线预测任务中高效和有效地工作:1) 一种新的查询设计,称为动态查询,将单个线条查询分解为可分离的点查询,可以直觉地提高回归任务的本地化准确性;2) 一种基于动态查询的逐步线回归方法,以逐步回归线上的点,进一步加强了扭曲表格的本地化准确性;3) 一种增强前向匹配策略,以解决 DETR 的缓慢收敛问题。在分离线预测后,使用一个简单的关系网络 based 细胞融合模块来恢复连通细胞。通过这些新技术,我们的 TSR Former 在多个基准数据集上实现了最先进的性能,包括 SciTSR、PubTabNet、WTW 和 FinTabNet。此外,我们还在一个更具挑战性的现实世界内部数据集上验证了我们方法的鲁棒性和高本地化准确性。
URL
https://arxiv.org/abs/2303.11615
