Abstract
This research presents and compares multiple approaches to automate the generation of literature reviews using several Natural Language Processing (NLP) techniques and retrieval-augmented generation (RAG) with a Large Language Model (LLM). The ever-increasing number of research articles provides a huge challenge for manual literature review. It has resulted in an increased demand for automation. Developing a system capable of automatically generating the literature reviews from only the PDF files as input is the primary objective of this research work. The effectiveness of several Natural Language Processing (NLP) strategies, such as the frequency-based method (spaCy), the transformer model (Simple T5), and retrieval-augmented generation (RAG) with Large Language Model (GPT-3.5-turbo), is evaluated to meet the primary objective. The SciTLDR dataset is chosen for this research experiment and three distinct techniques are utilized to implement three different systems for auto-generating the literature reviews. The ROUGE scores are used for the evaluation of all three systems. Based on the evaluation, the Large Language Model GPT-3.5-turbo achieved the highest ROUGE-1 score, 0.364. The transformer model comes in second place and spaCy is at the last position. Finally, a graphical user interface is created for the best system based on the large language model.
Abstract (translated)
这项研究介绍了并比较了使用多种自然语言处理（NLP）技术和检索增强生成（RAG）与大型语言模型（LLM）来自动生成文献综述的多个方法。不断增加的研究文章数量为手动撰写文献综述带来了巨大的挑战，从而增加了对自动化的需求。本研究工作的主要目标是开发一个系统，该系统仅需PDF文件作为输入即可自动生成文献综述。为了实现这一主要目标，评估了多种自然语言处理（NLP）策略的有效性，包括基于频率的方法（spaCy）、变换器模型（Simple T5），以及与大型语言模型（GPT-3.5-turbo）相结合的检索增强生成（RAG）。本研究实验选择了SciTLDR数据集，并利用三种不同的技术实现了三个不同系统以自动产生文献综述。使用ROUGE分数来评估所有三个系统的性能。根据评估结果，大型语言模型GPT-3.5-turbo获得了最高的ROUGE-1得分，为0.364。变换器模型排名第二，而spaCy则排在最后一位。最终，基于最佳的大型语言模型系统创建了一个图形用户界面。
URL
https://arxiv.org/abs/2411.18583
