Paper Reading AI Learner

Enhancing Temporal Knowledge Graph Forecasting with Large Language Models via Chain-of-History Reasoning

2024-02-22 08:51:39
Yuwei Xia, Ding Wang, Qiang Liu, Liang Wang, Shu Wu, Xiaoyu Zhang

Abstract

Temporal Knowledge Graph (TKG) forecasting aims to predict future facts based on given histories. Most recent graph-based models excel at capturing structural information within TKGs but lack semantic comprehension abilities. Nowadays, with the surge of LLMs, the LLM-based TKG prediction model has emerged. However, the existing LLM-based model exhibits three shortcomings: (1) It only focuses on the first-order history for prediction while ignoring high-order historical information, resulting in the provided information for LLMs being extremely limited. (2) LLMs struggle with optimal reasoning performance under heavy historical information loads. (3) For TKG prediction, the temporal reasoning capability of LLM alone is limited. To address the first two challenges, we propose Chain-of-History (CoH) reasoning which explores high-order histories step-by-step, achieving effective utilization of high-order historical information for LLMs on TKG prediction. To address the third issue, we design CoH as a paly-and-plug module to enhance the performance of graph-based models for TKG prediction. Extensive experiments on three datasets and backbones demonstrate the effectiveness of CoH.

Abstract (translated)

语义知识图(TKG)预测旨在预测基于给定历史的未来事实。最先进的基于图的模型在TKG中表现出色,但缺乏语义理解能力。如今,随着LLM的快速发展,LLM-based TKG预测模型应运而生。然而,现有的LLM-based模型存在三个缺陷:(1)它仅关注预测的第一层历史信息,而忽略了高层历史信息,导致LLM提供的信息非常有限。(2)LLM在大量历史信息负载下难以实现最优推理性能。(3)对于TKG预测,LLM单独的时空推理能力有限。为了应对前两个挑战,我们提出了链式历史(CoH)推理,该推理逐步探索高层历史信息,有效利用LLM在TKG预测中的高层历史信息。为了应对第三个问题,我们设计了一个CoH作为插件和模块来增强基于图模型的TKG预测性能。在三个数据集和硬件上的广泛实验证明CoH的有效性。

URL

https://arxiv.org/abs/2402.14382

PDF

https://arxiv.org/pdf/2402.14382.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot