Abstract
Relation tuple extraction from text is an important task for building knowledge bases. Recently, joint entity and relation extraction models have achieved very high F1 scores in this task. However, the experimental settings used by these models are restrictive and the datasets used in the experiments are not realistic. They do not include sentences with zero tuples (zero-cardinality). In this paper, we evaluate the state-of-the-art joint entity and relation extraction models in a more realistic setting. We include sentences that do not contain any tuples in our experiments. Our experiments show that there is significant drop ($\sim 10-15\%$ in one dataset and $\sim 6-14\%$ in another dataset) in their F1 score in this setting. We also propose a two-step modeling using a simple BERT-based classifier that leads to improvement in the overall performance of these models in this realistic experimental setup.
Abstract (translated)
从文本提取关系tuple是一项建立知识库的重要任务。最近,联合实体和关系提取模型在这项任务中取得了非常高的F1得分。然而,这些模型使用的实验设置是限制性的,并且实验中使用的数据集并不现实。它们不包括没有 tuple (零属性)的语句。在本文中,我们在一个更现实的环境中评估最先进的联合实体和关系提取模型。我们在实验中包括没有 tuple 的语句。我们的实验结果表明,在这些模型在这种现实环境中的F1得分上出现了显著下降(一个数据集下降 $sim 10-15\%$,另一个数据集下降 $sim 6-14\%$)。我们还提出了使用一个简单的BERT基于分类器的两个步骤建模方法,这有助于在这些模型在这种现实实验环境中的总体表现上提高。
URL
https://arxiv.org/abs/2302.09887
