Abstract
Foundational to the Chinese language and culture, Chinese characters encompass extraordinarily extensive and ever-expanding categories, with the latest Chinese GB18030-2022 standard containing 87,887 categories. The accurate recognition of this vast number of characters, termed mega-category recognition, presents a formidable yet crucial challenge for cultural heritage preservation and digital applications. Despite significant advances in Optical Character Recognition (OCR), mega-category recognition remains unexplored due to the absence of comprehensive datasets, with the largest existing dataset containing merely 16,151 categories. To bridge this critical gap, we introduce MegaHan97K, a mega-category, large-scale dataset covering an unprecedented 97,455 categories of Chinese characters. Our work offers three major contributions: (1) MegaHan97K is the first dataset to fully support the latest GB18030-2022 standard, providing at least six times more categories than existing datasets; (2) It effectively addresses the long-tail distribution problem by providing balanced samples across all categories through its three distinct subsets: handwritten, historical and synthetic subsets; (3) Comprehensive benchmarking experiments reveal new challenges in mega-category scenarios, including increased storage demands, morphologically similar character recognition, and zero-shot learning difficulties, while also unlocking substantial opportunities for future research. To the best of our knowledge, the MetaHan97K is likely the dataset with the largest classes not only in the field of OCR but may also in the broader domain of pattern recognition. The dataset is available at this https URL.
Abstract (translated)
中文语言和文化的基础是中国汉字，它们包含极其广泛且不断扩大的类别。最新的《中华人民共和国国家标准GB18030-2022》包含了多达87,887个字符类别。准确识别这些海量字符，即所谓的“大规模类别识别”，对文化遗产保护及数字应用提出了一个艰巨但至关重要的挑战。尽管光学字符识别（OCR）技术取得了显著进展，但由于缺乏全面的数据集支持，大规模类别识别尚未被深入探索，目前已知的最大数据集仅包含16,151个类别。为填补这一关键空白，我们推出了MegaHan97K，这是一个涵盖前所未有的97,455个汉字类别的大规模数据集。 我们的工作做出了三大主要贡献： （1）MegaHan97K是首个全面支持最新GB18030-2022标准的数据集，并且其类别数量至少为现有数据集的六倍； （2）通过提供包括手写、历史和合成三种不同子集在内的平衡样本，它有效解决了长尾分布问题； （3）全面基准测试实验揭示了大规模场景下的新挑战，如增加的存储需求、形态相似字符识别难度以及零样本学习障碍，并为未来研究开辟了巨大机会。 据我们所知，MetaHan97K可能是OCR领域乃至更广泛的模式识别领域的类别最多的数据集。该数据集可在以下网址获取：[https://example.com]（请将链接替换为您实际的数据集发布页面）。
URL
https://arxiv.org/abs/2506.04807
