Abstract
Hierarchical text classification (HTC) is a natural language processing task which has the objective of categorising text documents into a set of classes from a predefined structured class hierarchy. Recent HTC approaches use various techniques to incorporate the hierarchical class structure information with the natural language understanding capabilities of pre-trained language models (PLMs) to improve classification performance. Furthermore, using topic models along with PLMs to extract features from text documents has been shown to be an effective approach for multi-label text classification tasks. The rationale behind the combination of these feature extractor models is that the PLM captures the finer-grained contextual and semantic information while the topic model obtains high-level representations which consider the corpus of documents as a whole. In this paper, we use a HTC approach which uses a PLM and a topic model to extract features from text documents which are used to train a classification model. Our objective is to determine whether the combination of the features extracted from the two models is beneficial to HTC performance in general. In our approach, the extracted features are passed through separate convolutional layers whose outputs are combined and passed to a label-wise attention mechanisms which obtains label-specific document representations by weighing the most important features for each class separately. We perform comprehensive experiments on three HTC benchmark datasets and show that using the features extracted from the topic model generally decreases classification performance compared to only using the features obtained by the PLM. In contrast to previous work, this shows that the incorporation of features extracted from topic models for text classification tasks should not be assumed beneficial.
Abstract (translated)
层次文本分类（HTC）是一种自然语言处理任务，其目标是将文本文档归类到预定义的结构化类别层级中的一个或多个类别中。近期的一些HTC方法采用各种技术来整合层次类别结构信息与预先训练的语言模型（PLM）的自然语言理解能力，以提高分类性能。此外，结合使用主题模型和PLM从文本文档中提取特征已被证明是多标签文本分类任务的有效方法。这种组合特征提取模型背后的理由是：PLM能够捕捉到更细致的上下文和语义信息，而主题模型则获取考虑整个文档集合的整体高级表示。 在本文中，我们使用了一种HTC方法，该方法利用一个PLM和一个主题模型从文本文档中提取特征，并用这些特征训练分类模型。我们的目标是确定将这两种模型提取的特征结合起来是否一般而言对HTC性能有益。在我们的方法中，提取到的特征通过各自的卷积层进行处理，然后将其输出组合并通过标签级别的注意力机制获取特定于每个类别的文档表示，该机制根据每种类型的需要为最重要的特征加权。 我们针对三个HTC基准数据集进行了全面实验，并显示使用主题模型提取的特征通常会降低分类性能，而仅仅利用PLM获得的特征则不会。与之前的研究不同的是，这表明在文本分类任务中采用从主题模型提取的特征不应被默认认为是有益的做法。
URL
https://arxiv.org/abs/2507.16490
