Abstract
The escalating volume of collected healthcare textual data presents a unique challenge for automated Multi-Label Text Classification (MLTC), which is primarily due to the scarcity of annotated texts for training and their nuanced nature. Traditional machine learning models often fail to fully capture the array of expressed topics. However, Large Language Models (LLMs) have demonstrated remarkable effectiveness across numerous Natural Language Processing (NLP) tasks in various domains, which show impressive computational efficiency and suitability for unsupervised learning through prompt engineering. Consequently, these LLMs promise an effective MLTC of medical narratives. However, when dealing with various labels, different prompts can be relevant depending on the topic. To address these challenges, the proposed approach, QUAD-LLM-MLTC, leverages the strengths of four LLMs: GPT-4o, BERT, PEGASUS, and BART. QUAD-LLM-MLTC operates in a sequential pipeline in which BERT extracts key tokens, PEGASUS augments textual data, GPT-4o classifies, and BART provides topics' assignment probabilities, which results in four classifications, all in a 0-shot setting. The outputs are then combined using ensemble learning and processed through a meta-classifier to produce the final MLTC result. The approach is evaluated using three samples of annotated texts, which contrast it with traditional and single-model methods. The results show significant improvements across the majority of the topics in the classification's F1 score and consistency (F1 and Micro-F1 scores of 78.17% and 80.16% with standard deviations of 0.025 and 0.011, respectively). This research advances MLTC using LLMs and provides an efficient and scalable solution to rapidly categorize healthcare-related text data without further training.
Abstract (translated)
收集到的医疗文本数据量的不断增加，为自动化多标签文本分类（MLTC）带来了独特的挑战。这一问题主要源于训练用注释文本的稀缺及其复杂性。传统的机器学习模型往往难以充分捕捉表达的主题范围。然而，在各种领域内，大型语言模型（LLMs）在众多自然语言处理（NLP）任务中展现了显著的效果，这些模型具有出色的计算效率，并且通过提示工程进行无监督学习表现出良好的适应性。因此，这些LLM为医疗叙事的MLTC提供了有效的解决方案。然而，在面对不同标签时，根据话题的不同，不同的提示可能是相关的。为了应对这些挑战，提出的QUAD-LLM-MLTC方法利用了四种大型语言模型（LLMs）的优势：GPT-4o、BERT、PEGASUS和BART。在该方法中，采用顺序流水线操作：首先由BERT提取关键标记，然后PEGASUS增强文本数据，接着GPT-4o进行分类，最后BART提供主题分配的概率，从而产生四类分类结果，并且所有这些都在零样本设置下完成（即无需进一步训练）。随后，通过集成学习将输出结合起来并通过元分类器处理以生成最终的MLTC结果。该方法利用三个注释文本样本进行了评估，将其与传统方法和单一模型方法进行对比。结果显示，在分类F1评分以及一致性方面，大多数话题都表现出显著改进（F1评分为78.17%，Micro-F1评分为80.16%，标准偏差分别为0.025和0.011）。这项研究推进了利用LLM进行MLTC的方法，并提供了一种有效且可扩展的解决方案，可以在无需进一步训练的情况下迅速分类医疗相关的文本数据。
URL
https://arxiv.org/abs/2502.14189
