Abstract
Extracting precise geographical information from textual contents is crucial in a plethora of applications. For example, during hazardous events, a robust and unbiased toponym extraction framework can provide an avenue to tie the location concerned to the topic discussed by news media posts and pinpoint humanitarian help requests or damage reports from social media. Early studies have leveraged rule-based, gazetteer-based, deep learning, and hybrid approaches to address this problem. However, the performance of existing tools is deficient in supporting operations like emergency rescue, which relies on fine-grained, accurate geographic information. The emerging pretrained language models can better capture the underlying characteristics of text information, including place names, offering a promising pathway to optimize toponym recognition to underpin practical applications. In this paper, TopoBERT, a toponym recognition module based on a one dimensional Convolutional Neural Network (CNN1D) and Bidirectional Encoder Representation from Transformers (BERT), is proposed and fine-tuned. Three datasets (CoNLL2003-Train, Wikipedia3000, WNUT2017) are leveraged to tune the hyperparameters, discover the best training strategy, and train the model. Another two datasets (CoNLL2003-Test and Harvey2017) are used to evaluate the performance. Three distinguished classifiers, linear, multi-layer perceptron, and CNN1D, are benchmarked to determine the optimal model architecture. TopoBERT achieves state-of-the-art performance (f1-score=0.865) compared to the other five baseline models and can be applied to diverse toponym recognition tasks without additional training.
Abstract (translated)
从文本内容提取精确的地理位置信息是众多应用的关键。例如,在危险事件中,一个稳定且无偏见的地名提取框架可以提供将相关位置与新闻媒体发布的话题以及社交媒体上的人道主义援助请求或损坏报告定位的途径。早期研究利用了规则、地名数据库、深度学习和混合方法来解决这一问题。然而,现有工具的性能在支持救援等精细、准确的地理信息依赖的工作方面表现不佳。新兴预训练语言模型能够更好地捕捉文本信息中包括地名的深层次特征,提供了优化地名识别以支持实际应用程序的有前途的途径。在本文中,提出了TopoBERT,一个基于一维卷积神经网络(CNN1D)和Transformers的双向编码表示的地名识别模块,并进行优化。三个数据集(CoNLL2003-训练、Wikipedia3000、WNUT2017)用于调整超参数、发现最佳训练策略并训练模型。另外两个数据集(CoNLL2003-测试和 Harvey2017)用于评估性能。三个区别明显的分类器、线性、多层感知器和CNN1D作为基准来确定最佳模型架构。TopoBERT相对于其他五个基准模型实现了先进的性能(f1得分=0.865),而且无需额外的训练可以应用于各种地名识别任务。
URL
https://arxiv.org/abs/2301.13631
