Abstract
Long text understanding is important yet challenging for natural language processing. A long article or document usually contains many redundant words that are not pertinent to its gist and sometimes can be regarded as noise. With recent advances of abstractive summarization, we propose our \emph{Gist Detector} to leverage the gist detection ability of a summarization model and integrate the extracted gist into downstream models to enhance their long text understanding ability. Specifically, Gist Detector first learns the gist detection knowledge distilled from a summarization model, and then produces gist-aware representations to augment downstream models. We evaluate our method on three different tasks: long document classification, distantly supervised open-domain question answering, and non-parallel text style transfer. The experimental results show that our method can significantly improve the performance of baseline models on all tasks.
Abstract (translated)
长文本理解对于自然语言处理来说很重要，但也具有挑战性。一篇长文章或文档通常包含许多与文章主旨不相关的冗余单词，有时可以被视为噪音。随着最近摘要性总结的进步，我们提出了我们的\emph{长文本理解检测器}，利用摘要模型的摘要检测能力，并将在下游模型中整合提取的摘要以提高其长文本理解能力。具体来说，Gist Detector首先从摘要模型中学习摘要检测知识，然后产生具有摘要意识的表示来增强下游模型。我们在三个不同的任务上评估我们的方法：长文档分类、远离监督的问题回答和非平行文本风格转移。实验结果表明，我们的方法可以在所有任务上显著提高基线模型的性能。
URL
https://arxiv.org/abs/2405.04955
