Abstract
Large language models (LLMs) suffer from the hallucination problem and face significant challenges when applied to knowledge-intensive tasks. A promising approach is to leverage evidence documents as extra supporting knowledge, which can be obtained through retrieval or generation. However, existing methods directly leverage the entire contents of the evidence document, which may introduce noise information and impair the performance of large language models. To tackle this problem, we propose a novel Knowledge Selection of Large Language Models (KS-LLM) method, aiming to identify valuable information from evidence documents. The KS-LLM approach utilizes triples to effectively select knowledge snippets from evidence documents that are beneficial to answering questions. Specifically, we first generate triples based on the input question, then select the evidence sentences most similar to triples from the evidence document, and finally combine the evidence sentences and triples to assist large language models in generating answers. Experimental comparisons on several question answering datasets, such as TriviaQA, WebQ, and NQ, demonstrate that the proposed method surpasses the baselines and achieves the best results.
Abstract (translated)
大语言模型（LLMs）在知识密集型任务中存在幻觉问题，并且当应用于知识密集型任务时，面临着显著的挑战。一个有前途的方法是利用证据文档作为额外的支持知识，这是通过检索或生成获得的。然而，现有的方法直接利用证据文档的整个内容，这可能会引入噪声信息并损害大型语言模型的性能。为解决这个问题，我们提出了一种名为知识选择大型语言模型（KS-LLM）的方法，旨在从证据文档中识别有价值的信息。KS-LLM方法利用三元组有效地选择对回答问题有益的证据句子。具体来说，我们首先根据输入问题生成三元组，然后从证据文档中选择与三元组最相似的证据句子，最后将证据句子和三元组结合以帮助大型语言模型生成答案。在多个问题回答数据集（如TriviaQA、WebQ和NQ）上的实验比较表明，与基线相比，所提出的方法超过了基线，并取得了最佳结果。
URL
https://arxiv.org/abs/2404.15660
