Abstract
Graphs play an important role in representing complex relationships in various domains like social networks, knowledge graphs, and molecular discovery. With the advent of deep learning, Graph Neural Networks (GNNs) have emerged as a cornerstone in Graph Machine Learning (Graph ML), facilitating the representation and processing of graph structures. Recently, LLMs have demonstrated unprecedented capabilities in language tasks and are widely adopted in a variety of applications such as computer vision and recommender systems. This remarkable success has also attracted interest in applying LLMs to the graph domain. Increasing efforts have been made to explore the potential of LLMs in advancing Graph ML's generalization, transferability, and few-shot learning ability. Meanwhile, graphs, especially knowledge graphs, are rich in reliable factual knowledge, which can be utilized to enhance the reasoning capabilities of LLMs and potentially alleviate their limitations such as hallucinations and the lack of explainability. Given the rapid progress of this research direction, a systematic review summarizing the latest advancements for Graph ML in the era of LLMs is necessary to provide an in-depth understanding to researchers and practitioners. Therefore, in this survey, we first review the recent developments in Graph ML. We then explore how LLMs can be utilized to enhance the quality of graph features, alleviate the reliance on labeled data, and address challenges such as graph heterogeneity and out-of-distribution (OOD) generalization. Afterward, we delve into how graphs can enhance LLMs, highlighting their abilities to enhance LLM pre-training and inference. Furthermore, we investigate various applications and discuss the potential future directions in this promising field.
Abstract (translated)
图在表示复杂关系方面在社交网络、知识图谱和分子发现等领域中发挥着重要作用。随着深度学习的出现，图神经网络（GNNs）成为图机器学习（Graph ML）的一个支柱，推动了图结构的表示和处理。近年来，LLM在语言任务上的表现已经达到了史无前例的水平，并在各种应用领域（如计算机视觉和推荐系统）得到了广泛应用。这一显著的成功也引起了将LLM应用于图形领域的兴趣。越来越多的努力致力于探索LLM在推动图机器学习的一般化、可迁移性和少样本学习能力方面的潜力。同时，特别是知识图谱，图形在可靠的事实知识方面非常丰富，可以利用来增强LLM的推理能力，并可能减轻其局限性，如幻觉和缺乏可解释性。鉴于这一研究领域的快速进步，对于LLM时代图机器学习的系统综述总结最新的进展是必要的，以提供研究人员和实践者对这一领域的深入理解。因此，在本次调查中，我们首先回顾了图机器学习领域的最新发展。然后，我们探讨了LLM如何用于提高图形特征的质量、减轻对标注数据的依赖以及解决诸如图形异质性和离散（OOD）泛化等问题。接着，我们深入研究了图形如何增强LLM，强调了它们在提高LLM预训练和推理能力方面的能力。最后，我们调查了各种应用，并讨论了这一充满前景的领域未来的潜在方向。
URL
https://arxiv.org/abs/2404.14928
