Abstract
Large Language Models (LLMs) have catalyzed significant advancements in Natural Language Processing (NLP), yet they encounter challenges such as hallucination and the need for domain-specific knowledge. To mitigate these, recent methodologies have integrated information retrieved from external resources with LLMs, substantially enhancing their performance across NLP tasks. This survey paper addresses the absence of a comprehensive overview on Retrieval-Augmented Language Models (RALMs), both Retrieval-Augmented Generation (RAG) and Retrieval-Augmented Understanding (RAU), providing an in-depth examination of their paradigm, evolution, taxonomy, and applications. The paper discusses the essential components of RALMs, including Retrievers, Language Models, and Augmentations, and how their interactions lead to diverse model structures and applications. RALMs demonstrate utility in a spectrum of tasks, from translation and dialogue systems to knowledge-intensive applications. The survey includes several evaluation methods of RALMs, emphasizing the importance of robustness, accuracy, and relevance in their assessment. It also acknowledges the limitations of RALMs, particularly in retrieval quality and computational efficiency, offering directions for future research. In conclusion, this survey aims to offer a structured insight into RALMs, their potential, and the avenues for their future development in NLP. The paper is supplemented with a Github Repository containing the surveyed works and resources for further study: this https URL.
Abstract (translated)
大规模语言模型（LLMs）在自然语言处理（NLP）领域催生了许多显著的进步，但它们仍然面临诸如幻觉和需要领域特定知识等挑战。为了缓解这些挑战，最近的方法将外部资源中检索到的信息与LLM相结合，极大地提高了它们在NLP任务上的表现。 这份调查论文讨论了关于检索增强语言模型（RALMs）的全面概述的缺失，包括检索增强生成（RAG）和检索增强理解（RAU），深入研究了它们的范式、演变、分类和应用。论文讨论了RALMs的重要组成部分，包括检索器、语言模型和增强器，以及它们之间的互动导致的不同模型结构和应用。RALMs在翻译和对话系统以及知识密集型应用等领域表现出实际价值。 调查包括对RALMs的几个评估方法的讨论，强调了它们的稳健性、准确性和相关性在评估中的重要性。它还承认了RALMs的局限性，特别是检索质量和计算效率方面的限制，为未来的研究提供了方向。总之，这份调查试图提供一个结构化的了解RALMs、它们的潜力和在NLP领域未来发展的途径。论文还附带了一个Github存储库，其中包含调查的工作和资源：https://github.com/。
URL
https://arxiv.org/abs/2404.19543
