Abstract
Handling graph data is one of the most difficult tasks. Traditional techniques, such as those based on geometry and matrix factorization, rely on assumptions about the data relations that become inadequate when handling large and complex graph data. On the other hand, deep learning approaches demonstrate promising results in handling large graph data, but they often fall short of providing interpretable explanations. To equip the graph processing with both high accuracy and explainability, we introduce a novel approach that harnesses the power of a large language model (LLM), enhanced by an uncertainty-aware module to provide a confidence score on the generated answer. We experiment with our approach on two graph processing tasks: few-shot knowledge graph completion and graph classification. Our results demonstrate that through parameter efficient fine-tuning, the LLM surpasses state-of-the-art algorithms by a substantial margin across ten diverse benchmark datasets. Moreover, to address the challenge of explainability, we propose an uncertainty estimation based on perturbation, along with a calibration scheme to quantify the confidence scores of the generated answers. Our confidence measure achieves an AUC of 0.8 or higher on seven out of the ten datasets in predicting the correctness of the answer generated by LLM.
Abstract (translated)
处理图数据是其中最具挑战性的任务之一。传统技术（如基于几何和矩阵分解的技术）依赖于处理大型和复杂图数据时的数据关系假设。另一方面，深度学习方法在处理大型图数据方面表现出有希望的结果，但它们通常无法提供可解释的解释。为了使图处理具有高准确性和可解释性，我们引入了一种名为LLM的新颖方法，该方法利用了大型语言模型的力量，并通过不确定性感知模块在生成的答案上提供置信分数。我们在两个图处理任务上进行实验：少样本知识图补全和图分类。我们的结果表明，通过参数高效的微调，LLM在十个不同的基准数据集上超越了最先进的算法。此外，为了应对可解释性的挑战，我们提出了基于扰动的不确定性估计和置信度校准方案。我们的置信度度量在预测由LLM生成的答案的正确性方面实现了AUC 0.8或更高。在预测正确性的数据上，有7个数据集的AUC在0.8或更高。
URL
https://arxiv.org/abs/2404.00589
