Abstract
We developed a proof-of-concept method for the automatic analysis of the structure and content of incunabula pages. A custom dataset comprising 500 annotated pages from five different incunabula was created using resources from the Jagiellonian Digital Library. Each page was manually labeled with five predefined classes: Text, Title, Picture, Table, and Handwriting. Additionally, the publicly available DocLayNet dataset was utilized as supplementary training data. To perform object detection, YOLO11n and YOLO11s models were employed and trained using two strategies: a combined dataset (DocLayNet and the custom dataset) and the custom dataset alone. The highest performance (F1 = 0.94) was achieved by the YOLO11n model trained exclusively on the custom data. Optical character recognition was then conducted on regions classified as Text, using both Tesseract and Kraken OCR, with Tesseract demonstrating superior results. Subsequently, image classification was applied to the Picture class using a ResNet18 model, achieving an accuracy of 98.7% across five subclasses: Decorative_letter, Illustration, Other, Stamp, and Wrong_detection. Furthermore, the CLIP model was utilized to generate semantic descriptions of illustrations. The results confirm the potential of machine learning in the analysis of early printed books, while emphasizing the need for further advancements in OCR performance and visual content interpretation.
Abstract (translated)
我们开发了一种概念验证方法，用于自动分析古籍印刷品页面的结构和内容。使用来自雅盖隆数字图书馆（Jagiellonian Digital Library）的资源创建了一个包含500页注释数据集，这些注释来自于五本不同的古登堡圣经，每一页都被手动标注为五个预定义类别之一：文本、标题、图片、表格和手写。此外，还使用了公开可用的DocLayNet数据集作为辅助训练数据。 为了执行目标检测任务，我们采用了YOLO11n和YOLO11s模型，并通过两种策略对其进行训练：一种是结合使用的数据集（DocLayNet和自定义数据集），另一种则是仅使用自定义数据集。结果显示，专门在自定义数据集上进行训练的YOLO11n模型取得了最佳性能（F1分数为0.94）。 随后，我们对被分类为“文本”的区域进行了光学字符识别（OCR）。这项任务采用了Tesseract和Kraken两种OCR工具，其中Tesseract表现更优。接着，我们使用ResNet18模型对图片类别进行图像分类，在五种子类（装饰字母、插图、其他、印章以及错误检测）中实现了98.7%的准确率。最后，还利用了CLIP模型生成关于插图的语义描述。 这些结果证实了机器学习在分析早期印刷书籍方面的潜力，同时也强调了需要进一步提高OCR性能和视觉内容解读方面的发展需求。
URL
https://arxiv.org/abs/2506.18069
