Paper Reading AI Learner

Label Informed Contrastive Pretraining for Node Importance Estimation on Knowledge Graphs

2024-02-26 12:28:51
Tianyu Zhang, Chengbin Hou, Rui Jiang, Xuegong Zhang, Chenghu Zhou, Ke Tang, Hairong Lv

Abstract

Node Importance Estimation (NIE) is a task of inferring importance scores of the nodes in a graph. Due to the availability of richer data and knowledge, recent research interests of NIE have been dedicating to knowledge graphs for predicting future or missing node importance scores. Existing state-of-the-art NIE methods train the model by available labels, and they consider every interested node equally before training. However, the nodes with higher importance often require or receive more attention in real-world scenarios, e.g., people may care more about the movies or webpages with higher importance. To this end, we introduce Label Informed ContrAstive Pretraining (LICAP) to the NIE problem for being better aware of the nodes with high importance scores. Specifically, LICAP is a novel type of contrastive learning framework that aims to fully utilize the continuous labels to generate contrastive samples for pretraining embeddings. Considering the NIE problem, LICAP adopts a novel sampling strategy called top nodes preferred hierarchical sampling to first group all interested nodes into a top bin and a non-top bin based on node importance scores, and then divide the nodes within top bin into several finer bins also based on the scores. The contrastive samples are generated from those bins, and are then used to pretrain node embeddings of knowledge graphs via a newly proposed Predicate-aware Graph Attention Networks (PreGAT), so as to better separate the top nodes from non-top nodes, and distinguish the top nodes within top bin by keeping the relative order among finer bins. Extensive experiments demonstrate that the LICAP pretrained embeddings can further boost the performance of existing NIE methods and achieve the new state-of-the-art performance regarding both regression and ranking metrics. The source code for reproducibility is available at this https URL

Abstract (translated)

节点重要性估计(NIE)是一个推断图中有节点重要性的任务。 由于可用数据的丰富性和知识的可用性,近年来 NIE 的研究兴趣已经转向知识图以预测未来的节点重要性分数。现有的 NIE 方法通过可用标签来训练模型,并且在训练之前认为每个感兴趣的节点同样重要。然而,具有较高重要性的节点通常在现实场景中需要或接收更多的关注,例如,人们可能更关注具有较高重要性的电影或网页。为此,我们引入了Label Informed ContrAstive Pretraining(LICAP)来解决 NIE 问题,以更好地了解具有高重要性的节点。具体来说,LICAP 是一种新颖的对比学习框架,旨在充分利用连续的标签来生成对比样本。考虑到 NIE 问题,LICAP 采用了一种新颖的采样策略,即根据节点重要性分数将所有感兴趣的节点分组到 top bin 和 non-top bin,然后根据分数将 top bin 内的节点进一步划分为多个较细的 bin。对比样本是从这些 bin 中生成的,然后通过一个新提出的预测者注意图注意力网络(PreGAT)用于预训练知识图的节点嵌入,以便更好地将 top 节点与非 top 节点区分开来,并在 top bin 内保持相对顺序。大量实验证明,LICAP 的预训练嵌入可以进一步提高现有 NIE 方法的性能,并在相关指标上实现新的最优性能。源代码可在此处访问:https:// this URL

URL

https://arxiv.org/abs/2402.17791

PDF

https://arxiv.org/pdf/2402.17791.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot