Abstract
Conventional approaches to relation extraction usually require a fixed set of pre-defined relations. Such requirement is hard to meet in many real applications, especially when new data and relations are emerging incessantly and it is computationally expensive to store all data and re-train the whole model every time new data and relations come in. We formulate such a challenging problem as lifelong relation extraction and investigate memory-efficient incremental learning methods without catastrophically forgetting knowledge learned from previous tasks. We first investigate a modified version of the stochastic gradient methods with a replay memory, which surprisingly outperforms recent state-of-the-art lifelong learning methods. We further propose to improve this approach to alleviate the forgetting problem by anchoring the sentence embedding space. Specifically, we utilize an explicit alignment model to mitigate the sentence embedding distortion of the learned model when training on new data and new relations. Experiment results on multiple benchmarks show that our proposed method significantly outperforms the state-of-the-art lifelong learning approaches.
Abstract (translated)
传统的关系提取方法通常需要一组固定的预定义关系。这种要求在许多实际应用中很难满足，特别是当新的数据和关系不断出现时，存储所有数据并在每次新的数据和关系出现时重新训练整个模型的计算代价很高。我们提出了一个具有挑战性的问题，如终身关系提取和研究记忆有效的增量学习方法，而没有灾难性地忘记从以前的任务中学习到的知识。我们首先研究了一个带有重放记忆的随机梯度方法的改进版本，它出人意料地优于最新的终身学习方法。我们还建议改进这种方法，通过锚定句子嵌入空间来缓解遗忘问题。具体地说，在新数据和新关系的训练中，我们使用一个显式对齐模型来减轻所学模型的句子嵌入失真。在多个基准上的实验结果表明，我们提出的方法明显优于最先进的终身学习方法。
URL
https://arxiv.org/abs/1903.02588
