Paper Reading AI Learner

Large Language Models as Topological Structure Enhancers for Text-Attributed Graphs

2023-11-24 07:53:48
Shengyin Sun, Yuxiang Ren, Chen Ma, Xuecang Zhang

Abstract

The latest advancements in large language models (LLMs) have revolutionized the field of natural language processing (NLP). Inspired by the success of LLMs in NLP tasks, some recent work has begun investigating the potential of applying LLMs in graph learning tasks. However, most of the existing work focuses on utilizing LLMs as powerful node feature augmenters, leaving employing LLMs to enhance graph topological structures an understudied problem. In this work, we explore how to leverage the information retrieval and text generation capabilities of LLMs to refine/enhance the topological structure of text-attributed graphs (TAGs) under the node classification setting. First, we propose using LLMs to help remove unreliable edges and add reliable ones in the TAG. Specifically, we first let the LLM output the semantic similarity between node attributes through delicate prompt designs, and then perform edge deletion and edge addition based on the similarity. Second, we propose using pseudo-labels generated by the LLM to improve graph topology, that is, we introduce the pseudo-label propagation as a regularization to guide the graph neural network (GNN) in learning proper edge weights. Finally, we incorporate the two aforementioned LLM-based methods for graph topological refinement into the process of GNN training, and perform extensive experiments on four real-world datasets. The experimental results demonstrate the effectiveness of LLM-based graph topology refinement (achieving a 0.15%--2.47% performance gain on public benchmarks).

Abstract (translated)

大语言模型(LLMs)的最新进展已经推动了自然语言处理(NLP)领域的发展。受到LLMs在NLP任务上的成功启发,一些最近的论文开始研究将LLMs应用于图学习任务的可能性。然而,现有的工作主要关注利用LLMs作为强大的节点特征增强器,而没有对使用LLMs增强图拓扑结构进行深入研究。在这项工作中,我们探讨了如何利用LLMs的信息检索和文本生成能力来优化/改善节点分类设置下的文本相关图(TAG)的拓扑结构。首先,我们提出使用LLMs帮助去除不可靠的边并添加可靠的边在TAG中。具体来说,我们首先让LLM通过精细提示设计输出节点属性的语义相似度,然后根据相似度进行边删除和添加。其次,我们提出使用LLM生成的伪标签来改善图拓扑结构,即引入伪标签传播作为指导图神经网络(GNN)学习适当边重量的正则化。最后,我们将基于LLM的两种拓扑结构优化方法纳入GNN训练过程,并在四个真实世界数据集上进行广泛的实验。实验结果表明,LLM基于拓扑结构优化(在公共基准测试上实现0.15%--2.47%的性能提升)的有效性。

URL

https://arxiv.org/abs/2311.14324

PDF

https://arxiv.org/pdf/2311.14324.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot