Abstract
Pre-trained language representation models achieve remarkable state of the art across a wide range of tasks in natural language processing. One of the latest advancements is BERT, a deep pre-trained transformer that yields much better results than its predecessors do. Despite its burgeoning popularity, however, BERT has not yet been applied to document classification. This task deserves attention, since it contains a few nuances: first, modeling syntactic structure matters less for document classification than for other problems, such as natural language inference and sentiment classification. Second, documents often have multiple labels across dozens of classes, which is uncharacteristic of the tasks that BERT explores. In this paper, we describe fine-tuning BERT for document classification. We are the first to demonstrate the success of BERT on this task, achieving state of the art across four popular datasets.
Abstract (translated)
经过预先培训的语言表示模型在自然语言处理的各种任务中都达到了显著的技术水平。最新的进步之一是伯特，一个深度预先训练的变压器，产生比其前任更好的结果。然而，尽管伯特的流行迅速，但它还没有被应用到文档分类中。这项任务值得注意，因为它包含一些细微的差别：首先，与其他问题（如自然语言推理和情感分类）相比，建模语法结构对文档分类的影响较小。第二，文档通常在几十个类中有多个标签，这与Bert所研究的任务不同。在本文中，我们描述了文档分类的微调伯特。我们是第一个证明伯特在这项任务上取得成功的人，在四个流行的数据集中实现了最先进的技术。
URL
https://arxiv.org/abs/1904.08398
