Traditional knowledge graph embedding (KGE) methods typically require preserving the entire knowledge graph (KG) with significant training costs when new knowledge emerges. To address this issue, the continual knowledge graph embedding (CKGE) task has been proposed to train the KGE model by learning emerging knowledge efficiently while simultaneously preserving decent old knowledge. However, the explicit graph structure in KGs, which is critical for the above goal, has been heavily ignored by existing CKGE methods. On the one hand, existing methods usually learn new triples in a random order, destroying the inner structure of new KGs. On the other hand, old triples are preserved with equal priority, failing to alleviate catastrophic forgetting effectively. In this paper, we propose a competitive method for CKGE based on incremental distillation (IncDE), which considers the full use of the explicit graph structure in KGs. First, to optimize the learning order, we introduce a hierarchical strategy, ranking new triples for layer-by-layer learning. By employing the inter- and intra-hierarchical orders together, new triples are grouped into layers based on the graph structure features. Secondly, to preserve the old knowledge effectively, we devise a novel incremental distillation mechanism, which facilitates the seamless transfer of entity representations from the previous layer to the next one, promoting old knowledge preservation. Finally, we adopt a two-stage training paradigm to avoid the over-corruption of old knowledge influenced by under-trained new knowledge. Experimental results demonstrate the superiority of IncDE over state-of-the-art baselines. Notably, the incremental distillation mechanism contributes to improvements of 0.2%-6.5% in the mean reciprocal rank (MRR) score.
传统知识图嵌入(KGE)方法通常需要在新的知识出现时付出巨大的训练成本来保留整个知识图(KG)。为解决这个问题,连续知识图嵌入(CKGE)任务被提出,通过在同时学习和保留旧知识的同时,以高效的方式训练KGE模型。然而,现有的CKGE方法对KGs的显式图结构的重大忽视。一方面,现有方法通常在学习过程中以随机顺序学习新的三元组,破坏了新KG的内部结构。另一方面,旧三元组以等同的优先级被保留,未能有效减轻灾难性遗忘。在本文中,我们提出了一个基于增量的 distillation(IncDE)的竞争性的CKGE 方法,该方法考虑了 KGs 的显式图结构的全面利用。首先,为了优化学习顺序,我们引入了一个层次策略,对每个层级的新的三元组进行排序。通过同时使用上下文和内部层次结构,将新的三元组分组到基于图结构特征的层中。其次,为了有效地保留旧知识,我们设计了一种新颖的增量式蒸馏机制,促进了从前一层到下一层的实体表示的平稳转移,促进了旧知识的保留。最后,我们采用两级训练范式来避免受到欠训练的新知识中旧知识的过度污染。实验结果表明,与最先进的基线相比,IncDE 的优越性得到了充分证明。值得注意的是,增量式蒸馏机制对平均互反排名(MRR)得分提高了0.2%-6.5%。
https://arxiv.org/abs/2405.04453
With the rapid expansion of academic literature and the proliferation of preprints, researchers face growing challenges in manually organizing and labeling large volumes of articles. The NSLP 2024 FoRC Shared Task I addresses this challenge organized as a competition. The goal is to develop a classifier capable of predicting one of 123 predefined classes from the Open Research Knowledge Graph (ORKG) taxonomy of research fields for a given article.This paper presents our results. Initially, we enrich the dataset (containing English scholarly articles sourced from ORKG and arXiv), then leverage different pre-trained language Models (PLMs), specifically BERT, and explore their efficacy in transfer learning for this downstream task. Our experiments encompass feature-based and fine-tuned transfer learning approaches using diverse PLMs, optimized for scientific tasks, including SciBERT, SciNCL, and SPECTER2. We conduct hyperparameter tuning and investigate the impact of data augmentation from bibliographic databases such as OpenAlex, Semantic Scholar, and Crossref. Our results demonstrate that fine-tuning pre-trained models substantially enhances classification performance, with SPECTER2 emerging as the most accurate model. Moreover, enriching the dataset with additional metadata improves classification outcomes significantly, especially when integrating information from S2AG, OpenAlex and Crossref. Our best-performing approach achieves a weighted F1-score of 0.7415. Overall, our study contributes to the advancement of reliable automated systems for scholarly publication categorization, offering a potential solution to the laborious manual curation process, thereby facilitating researchers in efficiently locating relevant resources.
https://arxiv.org/abs/2405.04136
This paper presents CleanGraph, an interactive web-based tool designed to facilitate the refinement and completion of knowledge graphs. Maintaining the reliability of knowledge graphs, which are grounded in high-quality and error-free facts, is crucial for real-world applications such as question-answering and information retrieval systems. These graphs are often automatically assembled from textual sources by extracting semantic triples via information extraction. However, assuring the quality of these extracted triples, especially when dealing with large or low-quality datasets, can pose a significant challenge and adversely affect the performance of downstream applications. CleanGraph allows users to perform Create, Read, Update, and Delete (CRUD) operations on their graphs, as well as apply models in the form of plugins for graph refinement and completion tasks. These functionalities enable users to enhance the integrity and reliability of their graph data. A demonstration of CleanGraph and its source code can be accessed at this https URL under the MIT License.
本文介绍了CleanGraph,一个交互式的网页工具,旨在促进知识图谱的完善和完成。保持知识图谱的可靠性,这些知识图谱基于高质量和无错误的事实,对于现实世界的应用,如问答和信息检索系统,至关重要。这些图通常通过提取语义三元组来自文本来源。然而,在处理大型或低质量数据集时,确保这些提取的三元组质量具有相当大的挑战,并会破坏下游应用的性能。CleanGraph允许用户执行创建、读取、更新和删除(CRUD)操作,以及以插件形式应用模型来完成知识图谱的完善和完成任务。这些功能使用户能够增强其图形数据的完整性和可靠性。CleanGraph及其源代码的演示地址可以在https://www.clean-graph.org/ under the MIT License中访问。
https://arxiv.org/abs/2405.03932
Integrating large language models (LLMs) and knowledge graphs (KGs) holds great promise for revolutionizing intelligent education, but challenges remain in achieving personalization, interactivity, and explainability. We propose FOKE, a Forest Of Knowledge and Education framework that synergizes foundation models, knowledge graphs, and prompt engineering to address these challenges. FOKE introduces three key innovations: (1) a hierarchical knowledge forest for structured domain knowledge representation; (2) a multi-dimensional user profiling mechanism for comprehensive learner modeling; and (3) an interactive prompt engineering scheme for generating precise and tailored learning guidance. We showcase FOKE's application in programming education, homework assessment, and learning path planning, demonstrating its effectiveness and practicality. Additionally, we implement Scholar Hero, a real-world instantiation of FOKE. Our research highlights the potential of integrating foundation models, knowledge graphs, and prompt engineering to revolutionize intelligent education practices, ultimately benefiting learners worldwide. FOKE provides a principled and unified approach to harnessing cutting-edge AI technologies for personalized, interactive, and explainable educational services, paving the way for further research and development in this critical direction.
集成大型语言模型(LLMs)和知识图(KGs)在颠覆智能教育方面具有巨大的潜力,但在实现个性化、互动性和可解释性方面仍然存在挑战。我们提出了FOKE,一种结合基础模型、知识图和提示工程的方法,以解决这些挑战。FOKE引入了三个关键创新:(1)用于表示结构化领域知识的分层知识森林;(2) comprehensive learner modeling 的多维度用户跟踪机制;(3)用于生成精确、定制化学习指导的交互式提示工程方案。我们在编程教育、作业评估和学习路径规划中展示了FOKE的应用,证明了其有效性和实用性。此外,我们还实现了Scholar Hero,一个基于FOKE的现实生活中实例。我们的研究突出了将基础模型、知识图和提示工程集成到智能教育实践中,以颠覆教育传统,最终为全球学习者带来利益的潜力。FOKE提供了一种理性和统一的方法,利用最先进的人工智能技术为个性化、互动性和可解释性教育服务,为这个关键领域的研究和开发铺平道路。
https://arxiv.org/abs/2405.03734
The rapid advancement in artificial intelligence (AI), particularly through deep neural networks, has catalyzed significant progress in fields such as vision and text processing. Nonetheless, the pursuit of AI systems that exhibit human-like reasoning and interpretability continues to pose a substantial challenge. The Neural-Symbolic paradigm, which integrates the deep learning prowess of neural networks with the reasoning capabilities of symbolic systems, presents a promising pathway toward developing more transparent and comprehensible AI systems. Within this paradigm, the Knowledge Graph (KG) emerges as a crucial element, offering a structured and dynamic method for representing knowledge through interconnected entities and relationships, predominantly utilizing the triple (subject, predicate, object). This paper explores recent advancements in neural-symbolic integration based on KG, elucidating how KG underpins this integration across three key categories: enhancing the reasoning and interpretability of neural networks through the incorporation of symbolic knowledge (Symbol for Neural), refining the completeness and accuracy of symbolic systems via neural network methodologies (Neural for Symbol), and facilitating their combined application in Hybrid Neural-Symbolic Integration. It highlights current trends and proposes directions for future research in the domain of Neural-Symbolic AI.
人工智能(AI)的快速发展,特别是通过深度神经网络,在视觉和文本处理等领域取得了显著的进步。然而,追求具有类人推理和可解释性的AI系统仍然是一个巨大的挑战。神经符号范式将神经网络的深度学习能力与符号系统的推理能力相结合,为开发更透明和可解释的AI系统提供了有益的途径。在这种范式中,知识图(KG)成为了一个关键要素,它通过连接实体和关系提供了一个结构化和动态的方法来表示知识,主要利用三元组(主体,谓词,对象)。本文探讨了基于KG的神经符号整合最近的研究进展,解释了KG如何通过引入符号知识(Symbol for Neural)来提高神经网络的推理和可解释性,通过神经网络方法论(Neural for Symbol)来优化符号系统的完整性准确性,并通过混合神经-符号整合来促进它们的联合应用。它强调了当前领域内的趋势,并提出了未来在神经符号AI领域的研究方向。
https://arxiv.org/abs/2405.03524
Image-based retrieval in large Earth observation archives is challenging because one needs to navigate across thousands of candidate matches only with the query image as a guide. By using text as information supporting the visual query, the retrieval system gains in usability, but at the same time faces difficulties due to the diversity of visual signals that cannot be summarized by a short caption only. For this reason, as a matching-based task, cross-modal text-image retrieval often suffers from information asymmetry between texts and images. To address this challenge, we propose a Knowledge-aware Text-Image Retrieval (KTIR) method for remote sensing images. By mining relevant information from an external knowledge graph, KTIR enriches the text scope available in the search query and alleviates the information gaps between texts and images for better matching. Moreover, by integrating domain-specific knowledge, KTIR also enhances the adaptation of pre-trained vision-language models to remote sensing applications. Experimental results on three commonly used remote sensing text-image retrieval benchmarks show that the proposed knowledge-aware method leads to varied and consistent retrievals, outperforming state-of-the-art retrieval methods.
大地球观测档案中基于图像的检索具有挑战性,因为需要仅以查询图像为指南穿越数千个候选匹配。通过将文本作为支持视觉查询的信息,检索系统在可用性方面获得了提高,但同时由于视觉信号的多样性无法仅通过短文标题来总结,因此面临着困难。因此,作为一种匹配为基础的任务,跨模态文本-图像检索常常存在文本和图像之间的信息不对称。为了应对这一挑战,我们提出了一个知识引导的文本-图像检索(KTIR)方法来解决遥感图像。通过从外部知识图中挖掘相关信息,KTIR为搜索查询提供了更丰富的文本范围,并减轻了文本和图像之间的信息缺口,从而实现更好的匹配。此外,通过整合领域特定知识,KTIR还增强了预训练视觉语言模型对远程观测应用的适应性。在三个常用的遥感文本-图像检索基准测试中,与最先进的检索方法相比,所提出的知识引导方法产生了各种不同的检索结果,但均具有更好的表现。
https://arxiv.org/abs/2405.03373
Knowledge Graphs have been widely used to represent facts in a structured format. Due to their large scale applications, knowledge graphs suffer from being incomplete. The relation prediction task obtains knowledge graph completion by assigning one or more possible relations to each pair of nodes. In this work, we make use of the knowledge graph node names to fine-tune a large language model for the relation prediction task. By utilizing the node names only we enable our model to operate sufficiently in the inductive settings. Our experiments show that we accomplish new scores on a widely used knowledge graph benchmark.
知识图谱已被广泛用于以结构化格式表示事实。由于其大规模应用,知识图谱存在不完整性。关系预测任务通过为每对节点分配一个或多个可能的关系来获得知识图的完成。在这项工作中,我们利用知识图节点名称来微调一个大型语言模型,用于关系预测任务。通过仅使用节点名称,使我们模型能够在归纳设置中充分操作。我们的实验结果表明,我们在广泛使用的知识图基准上实现了新的分数。
https://arxiv.org/abs/2405.02738
Knowledge graphs have emerged as a sophisticated advancement and refinement of semantic networks, and their deployment is one of the critical methodologies in contemporary artificial intelligence. The construction of knowledge graphs is a multifaceted process involving various techniques, where researchers aim to extract the knowledge from existing resources for the construction since building from scratch entails significant labor and time costs. However, due to the pervasive issue of heterogeneity, the description diversity across different knowledge graphs can lead to mismatches between concepts, thereby impacting the efficacy of knowledge extraction. This Ph.D. study focuses on automatic knowledge graph extension, i.e., properly extending the reference knowledge graph by extracting and integrating concepts from one or more candidate knowledge graphs. We propose a novel knowledge graph extension framework based on entity type recognition. The framework aims to achieve high-quality knowledge extraction by aligning the schemas and entities across different knowledge graphs, thereby enhancing the performance of the extension. This paper elucidates three major contributions: (i) we propose an entity type recognition method exploiting machine learning and property-based similarities to enhance knowledge extraction; (ii) we introduce a set of assessment metrics to validate the quality of the extended knowledge graphs; (iii) we develop a platform for knowledge graph acquisition, management, and extension to benefit knowledge engineers practically. Our evaluation comprehensively demonstrated the feasibility and effectiveness of the proposed extension framework and its functionalities through quantitative experiments and case studies.
知识图谱已成为语义网络的先进发展和精炼,其在当代人工智能中扮演着关键方法论的角色。知识图谱的构建是一个多方面的过程,涉及各种技术,研究人员旨在从现有资源中提取知识,因为从头开始构建会带来大量的人力和时间成本。然而,由于普遍存在的异质性问题,知识图谱之间的描述差异可能导致概念之间的不匹配,从而影响知识提取的效力。本博士学位论文专注于自动知识图谱扩展,即通过提取和整合一个或多个候选知识图谱来扩展参考知识图谱。我们提出了一个基于实体类型识别的知识图谱扩展框架。该框架旨在通过将不同知识图谱之间的模式和实体对齐,实现高质量的知识扩展,从而提高扩展的性能。本文阐明了三个主要贡献: (i)我们提出了一种利用机器学习和属性基于相似性的实体类型识别方法,以增强知识提取; (ii)我们引入了一组评估指标来验证扩展知识图谱的质量; (iii)我们开发了一个知识图谱获取、管理和扩展平台,以帮助知识工程师实际操作。我们的评估全面展示了所提出的扩展框架的可行性和有效性,以及其实用性。
https://arxiv.org/abs/2405.02463
Structured science summaries or research contributions using properties or dimensions beyond traditional keywords enhances science findability. Current methods, such as those used by the Open Research Knowledge Graph (ORKG), involve manually curating properties to describe research papers' contributions in a structured manner, but this is labor-intensive and inconsistent between the domain expert human curators. We propose using Large Language Models (LLMs) to automatically suggest these properties. However, it's essential to assess the readiness of LLMs like GPT-3.5, Llama 2, and Mistral for this task before application. Our study performs a comprehensive comparative analysis between ORKG's manually curated properties and those generated by the aforementioned state-of-the-art LLMs. We evaluate LLM performance through four unique perspectives: semantic alignment and deviation with ORKG properties, fine-grained properties mapping accuracy, SciNCL embeddings-based cosine similarity, and expert surveys comparing manual annotations with LLM outputs. These evaluations occur within a multidisciplinary science setting. Overall, LLMs show potential as recommendation systems for structuring science, but further finetuning is recommended to improve their alignment with scientific tasks and mimicry of human expertise.
使用超越传统关键词的属性或维度来结构化科学摘要或研究贡献可以提高科学可查找性。目前的方法,如Open Research Knowledge Graph (ORKG)中所使用的,需要手动编辑属性以描述研究论文的贡献,但这是劳动密集型且与领域专家人类编者之间存在不一致性。我们提出使用大型语言模型(LLMs)来自动建议这些属性。然而,在应用之前评估LLMs(如GPT-3.5、Llama 2和Mistral)的准备情况至关重要。 我们的研究对ORKG手动编辑的属性和上述最先进的LLM生成的属性进行了全面比较分析。我们通过四个独特的视角来评估LLM性能:语义对齐和与ORKG属性之间的偏移,细粒度属性映射准确度,基于SciNCL嵌入的余弦相似度,以及专家调查与LLM输出之间的比较。这些评估发生在多学科科学环境中。 总的来说,LLMs在构建科学推荐系统方面具有潜力,但需要进一步的微调以改善其与科学任务的同步性和模拟人类专业知识的能力。
https://arxiv.org/abs/2405.02105
Answering complex logical queries over incomplete knowledge graphs (KGs) is challenging. Most previous works have focused on learning entity/relation embeddings and simulating first-order logic operators with various neural networks. However, they are bottlenecked by the inability to share world knowledge to improve logical reasoning, thus resulting in suboptimal performance. In this paper, we propose a complex logical reasoning schema over knowledge graphs upon large language models (LLMs), containing a curriculum-based logical-aware instruction tuning framework, named LACT. Specifically, we augment the arbitrary first-order logical queries via binary tree decomposition, to stimulate the reasoning capability of LLMs. To address the difficulty gap among different types of complex queries, we design a simple and flexible logic-aware curriculum learning framework. Experiments across widely used datasets demonstrate that LACT has substantial improvements~(brings an average +5.5% MRR score) over advanced methods, achieving the new state-of-the-art. Our code and model will be released at GitHub and huggingface soon.
回答复杂逻辑查询在半完整知识图(KG)上具有挑战性。大多数先前的研究都专注于学习实体/关系嵌入和用各种神经网络模拟第一范式逻辑操作。然而,它们因无法共享世界知识来提高逻辑推理能力而陷入瓶颈,从而导致性能较低。在本文中,我们提出了一种在大型语言模型(LLMs)上的复杂逻辑推理模式,包含一个基于课程的逻辑感知指令调整框架,称为LACT。具体来说,我们通过二叉树分解来增强任意第一范式逻辑查询,以刺激LLMs的推理能力。为了解决不同类型复杂查询之间的困难差距,我们设计了一个简单而灵活的逻辑感知课程学习框架。在广泛使用数据集上的实验证明,LACT在先进方法上取得了很大的改进(将平均+5.5%的MRR得分)实现了一种新的最优状态。我们的代码和模型将在GitHub和huggingface不久后发布。
https://arxiv.org/abs/2405.01649
Prediction of road users' behaviors in the context of autonomous driving has gained considerable attention by the scientific community in the last years. Most works focus on predicting behaviors based on kinematic information alone, a simplification of the reality since road users are humans, and as such they are highly influenced by their surrounding context. In addition, a large plethora of research works rely on powerful Deep Learning techniques, which exhibit high performance metrics in prediction tasks but may lack the ability to fully understand and exploit the contextual semantic information contained in the road scene, not to mention their inability to provide explainable predictions that can be understood by humans. In this work, we propose an explainable road users' behavior prediction system that integrates the reasoning abilities of Knowledge Graphs (KG) and the expressiveness capabilities of Large Language Models (LLM) by using Retrieval Augmented Generation (RAG) techniques. For that purpose, Knowledge Graph Embeddings (KGE) and Bayesian inference are combined to allow the deployment of a fully inductive reasoning system that enables the issuing of predictions that rely on legacy information contained in the graph as well as on current evidence gathered in real time by onboard sensors. Two use cases have been implemented following the proposed approach: 1) Prediction of pedestrians' crossing actions; 2) Prediction of lane change maneuvers. In both cases, the performance attained surpasses the current state of the art in terms of anticipation and F1-score, showing a promising avenue for future research in this field.
近年来,自动驾驶背景下预测道路使用者的行为已经引起了科学界的广泛关注。大多数工作都基于运动信息预测行为,简化现实,因为道路使用者是是人,所以他们对周围环境的影响很大。此外,大量研究作品依赖强大的深度学习技术,在预测任务中表现出高的性能指标,但可能无法完全理解并利用道路场景中的上下文语义信息,更不用说无法提供可解释的预测,让人类能够理解。在本文中,我们提出了一个可解释的道路使用者行为预测系统,通过使用检索增强生成(RAG)技术将知识图谱的推理能力和大型语言模型的表现力相结合。为此,知识图谱嵌入(KGE)和贝叶斯推理被结合使用,以便部署一个完全归纳推理系统,该系统能够基于图形中的旧信息以及车载传感器实时收集的证据发出预测。以下是根据所提出的方法实现的两个用例:1)预测行人过马路的行为;2)预测车道变更操作。在这两个用例中,取得的性能已经超越了当前的技术水平,显示了该领域未来研究的希望。
https://arxiv.org/abs/2405.00449
Distinguished from traditional knowledge graphs (KGs), temporal knowledge graphs (TKGs) must explore and reason over temporally evolving facts adequately. However, existing TKG approaches still face two main challenges, i.e., the limited capability to model arbitrary timestamps continuously and the lack of rich inference patterns under temporal constraints. In this paper, we propose an innovative TKGE method (PTBox) via polynomial decomposition-based temporal representation and box embedding-based entity representation to tackle the above-mentioned problems. Specifically, we decompose time information by polynomials and then enhance the model's capability to represent arbitrary timestamps flexibly by incorporating the learnable temporal basis tensor. In addition, we model every entity as a hyperrectangle box and define each relation as a transformation on the head and tail entity boxes. The entity boxes can capture complex geometric structures and learn robust representations, improving the model's inductive capability for rich inference patterns. Theoretically, our PTBox can encode arbitrary time information or even unseen timestamps while capturing rich inference patterns and higher-arity relations of the knowledge base. Extensive experiments on real-world datasets demonstrate the effectiveness of our method.
与传统知识图(KGs)相比,时间知识图(TKGs)必须充分探索和推理随时间变化的事实。然而,现有的TKG方法仍然面临着两个主要挑战,即无法连续建模任意时间戳以及缺乏在时间约束下丰富的推理模式。在本文中,我们提出了一个创新的时间知识图(TKGE)方法(PTBox),通过基于多项式的时本表示和基于箱嵌入的实体表示来解决上述问题。具体来说,我们通过多项式分解来分解时间信息,然后通过学习的时间本张量增强模型的能力来表示任意时间戳。此外,我们将每个实体建模为一个超矩形框,将每个关系建模为一个变换,该变换作用于头和尾实体框。实体框可以捕捉复杂的几何结构,并学习稳健的表示,提高模型对于复杂推理模式的归纳能力。从理论上将我们的PTBox可以编码任意时间信息或甚至未知的时刻,同时捕捉知识库中的丰富推理模式和高阶关系。在现实世界的数据集上进行大量实验证明了我们方法的有效性。
https://arxiv.org/abs/2405.00358
Temporal Knowledge Graph (TKG) reasoning often involves completing missing factual elements along the timeline. Although existing methods can learn good embeddings for each factual element in quadruples by integrating temporal information, they often fail to infer the evolution of temporal facts. This is mainly because of (1) insufficiently exploring the internal structure and semantic relationships within individual quadruples and (2) inadequately learning a unified representation of the contextual and temporal correlations among different quadruples. To overcome these limitations, we propose a novel Transformer-based reasoning model (dubbed ECEformer) for TKG to learn the Evolutionary Chain of Events (ECE). Specifically, we unfold the neighborhood subgraph of an entity node in chronological order, forming an evolutionary chain of events as the input for our model. Subsequently, we utilize a Transformer encoder to learn the embeddings of intra-quadruples for ECE. We then craft a mixed-context reasoning module based on the multi-layer perceptron (MLP) to learn the unified representations of inter-quadruples for ECE while accomplishing temporal knowledge reasoning. In addition, to enhance the timeliness of the events, we devise an additional time prediction task to complete effective temporal information within the learned unified representation. Extensive experiments on six benchmark datasets verify the state-of-the-art performance and the effectiveness of our method.
时空知识图(TKG)推理通常涉及在时间轴上完成缺失的事实元素。尽管现有的方法可以通过整合时间信息来学习每个事实元素的较好嵌入,但它们往往无法推断不同四元组之间的时空事实的演变。这主要是因为(1)对每个四元组内部结构和语义关系内部探索不足;(2)对不同四元组之间的上下文和时间关联的统一表示学习不足。为了克服这些限制,我们提出了一个新颖的Transformer基推理模型(被称为ECEformer),用于TKG学习演化链事件(ECE)。具体来说,我们按时间顺序展开实体节点周围的子图,将演化链事件作为输入,为我们的模型。随后,我们利用Transformer编码器学习 intra-quadruples 的嵌入。然后,我们根据多层感知器(MLP)构建混合上下文推理模块,学习不同四元组之间的统一表示,实现时间知识推理。此外,为了增强事件的及时性,我们还设计了一个额外的时间预测任务,以完成所学习到的统一表示中的有效时间信息。在六个基准数据集上的大量实验证实了我们的方法具有最先进的表现和效果。
https://arxiv.org/abs/2405.00352
Data protection and privacy is becoming increasingly crucial in the digital era. Numerous companies depend on third-party vendors and service providers to carry out critical functions within their operations, encompassing tasks such as data handling and storage. However, this reliance introduces potential vulnerabilities, as these vendors' security measures and practices may not always align with the standards expected by regulatory bodies. Businesses are required, often under the penalty of law, to ensure compliance with the evolving regulatory rules. Interpreting and implementing these regulations pose challenges due to their complexity. Regulatory documents are extensive, demanding significant effort for interpretation, while vendor-drafted privacy policies often lack the detail required for full legal compliance, leading to ambiguity. To ensure a concise interpretation of the regulatory requirements and compliance of organizational privacy policy with said regulations, we propose a Large Language Model (LLM) and Semantic Web based approach for privacy compliance. In this paper, we develop the novel Privacy Policy Compliance Verification Knowledge Graph, PrivComp-KG. It is designed to efficiently store and retrieve comprehensive information concerning privacy policies, regulatory frameworks, and domain-specific knowledge pertaining to the legal landscape of privacy. Using Retrieval Augmented Generation, we identify the relevant sections in a privacy policy with corresponding regulatory rules. This information about individual privacy policies is populated into the PrivComp-KG. Combining this with the domain context and rules, the PrivComp-KG can be queried to check for compliance with privacy policies by each vendor against relevant policy regulations. We demonstrate the relevance of the PrivComp-KG, by verifying compliance of privacy policy documents for various organizations.
数据保护和隐私在数字时代变得越来越重要。许多公司依赖第三方供应商和提供商执行运营中的关键功能,包括数据处理和存储。然而,这种依赖会引入潜在的安全漏洞,因为这些供应商的安全措施和做法可能并不总是符合监管机构的期望标准。根据法律要求,企业必须确保符合不断变化的监管规则。解释和实施这些法规由于其复杂性而面临挑战。监管文件涵盖广泛,需要大量精力进行解释,而供应商编写的隐私政策往往缺乏所需的详细信息,导致模糊不清。为了确保对监管要求和组织隐私政策的一致解释和合规,我们提出了一个基于大型语言模型(LLM)和语义网络的隐私合规方法。在本文中,我们开发了新颖的隐私政策合规验证知识图谱,PrivComp-KG。该知识图谱旨在有效地存储和检索关于隐私政策、监管框架和隐私法律领域的全面信息。通过检索增强生成,我们识别隐私政策中相应的法规条文。将个人隐私政策的信息充实到PrivComp-KG中。结合领域上下文和规则,PrivComp-KG可以用于检查每个供应商是否符合相关政策法规。我们通过验证各种组织的隐私政策文件对隐私政策的一致性进行验证,展示了PrivComp-KG的相關性。
https://arxiv.org/abs/2404.19744
External knowledge graphs (KGs) can be used to augment large language models (LLMs), while simultaneously providing an explainable knowledge base of facts that can be inspected by a human. This approach may be particularly valuable in domains where explainability is critical, like human trafficking data analysis. However, creating KGs can pose challenges. KGs parsed from documents may comprise explicit connections (those directly stated by a document) but miss implicit connections (those obvious to a human although not directly stated). To address these challenges, this preliminary research introduces the GAME-KG framework, standing for "Gaming for Augmenting Metadata and Enhancing Knowledge Graphs." GAME-KG is a federated approach to modifying explicit as well as implicit connections in KGs by using crowdsourced feedback collected through video games. GAME-KG is shown through two demonstrations: a Unity test scenario from Dark Shadows, a video game that collects feedback on KGs parsed from US Department of Justice (DOJ) Press Releases on human trafficking, and a following experiment where OpenAI's GPT-4 is prompted to answer questions based on a modified and unmodified KG. Initial results suggest that GAME-KG can be an effective framework for enhancing KGs, while simultaneously providing an explainable set of structured facts verified by humans.
外部知识图(KGs)可以用于增强大型语言模型(LLMs),同时为人类提供可检查的事实知识库。这种方法在需要解释性的领域(如打击人口贩卖数据分析)可能特别有价值。然而,创建KGs可能带来挑战。从文档中解析的KGs可能包括明确的连接(直接由文档中陈述的连接),但也会错过隐性的连接(对人类明显可见,尽管不是直接陈述的连接)。为解决这些挑战,这项初步研究引入了GAME-KG框架,它代表“通过游戏增强元数据和增强知识图”。GAME-KG是一种通过使用通过视频游戏收集的民办公众反馈修改KGs中明确和隐性连接的联邦方法。GAME-KG通过两个演示来说明:一个来自《黑暗影子》的Unity测试场景,该游戏收集了关于美国司法部(DOJ) press releases中的人口贩卖KGs的反馈,另一个是OpenAI的GPT-4被要求根据修改后的KG回答问题的实验。初步结果表明,GAME-KG可以成为增强KGs的有效框架,同时为人类提供经过验证的结构化事实知识库。
https://arxiv.org/abs/2404.19729
Trajectory prediction in autonomous driving relies on accurate representation of all relevant contexts of the driving scene including traffic participants, road topology, traffic signs as well as their semantic relations to each other. Despite increased attention to this issue, most approaches in trajectory prediction do not consider all of these factors sufficiently. This paper describes a method SemanticFormer to predict multimodal trajectories by reasoning over a semantic traffic scene graph using a hybrid approach. We extract high-level information in the form of semantic meta-paths from a knowledge graph which is then processed by a novel pipeline based on multiple attention mechanisms to predict accurate trajectories. The proposed architecture comprises a hierarchical heterogeneous graph encoder, which can capture spatio-temporal and relational information across agents and between agents and road elements, and a predictor that fuses the different encodings and decodes trajectories with probabilities. Finally, a refinement module evaluates permitted meta-paths of trajectories and speed profiles to obtain final predicted trajectories. Evaluation of the nuScenes benchmark demonstrates improved performance compared to the state-of-the-art methods.
轨迹预测在自动驾驶中依赖于准确地描述驾驶场景中所有相关上下文的轨迹,包括驾驶员、道路拓扑、交通标志以及它们之间的语义关系。尽管对这个问题给予了越来越多的关注,但大多数轨迹预测方法都没有考虑到所有这些因素。本文描述了一种名为 SemanticFormer 的方法,通过在语义交通场景图上进行推理来预测多模态轨迹。我们从一个知识图中提取高级信息,然后通过一种新颖的管道处理该知识图,预测准确的轨迹。所提出的架构包括分层异质图编码器,可以捕捉到车辆和道路元素以及它们之间的时空关系,以及一个预测器,将不同的编码器和解码轨迹的概率相结合。最后,一个优化模块评估了轨迹和速度剖面的允许元路径,以获得最终的预测轨迹。对 nuScenes 基准的评估显示,与最先进的方法相比,性能有所提高。
https://arxiv.org/abs/2404.19379
Knowledge graphs (KGs) are large datasets with specific structures representing large knowledge bases (KB) where each node represents a key entity and relations amongst them are typed edges. Natural language queries formed to extract information from a KB entail starting from specific nodes and reasoning over multiple edges of the corresponding KG to arrive at the correct set of answer nodes. Traditional approaches of question answering on KG are based on (a) semantic parsing (SP), where a logical form (e.g., S-expression, SPARQL query, etc.) is generated using node and edge embeddings and then reasoning over these representations or tuning language models to generate the final answer directly, or (b) information-retrieval based that works by extracting entities and relations sequentially. In this work, we evaluate the capability of (LLMs) to answer questions over KG that involve multiple hops. We show that depending upon the size and nature of the KG we need different approaches to extract and feed the relevant information to an LLM since every LLM comes with a fixed context window. We evaluate our approach on six KGs with and without the availability of example-specific sub-graphs and show that both the IR and SP-based methods can be adopted by LLMs resulting in an extremely competitive performance.
知识图(KGs)是具有特定结构的大型数据集,代表大量知识库(KB),其中每个节点表示一个关键实体,它们之间的关系是类型化的边。自然语言查询从特定的节点开始,通过推理与相应知识库中的多个边之间的关系,到达正确的答案节点。传统基于KG的问答方法是基于(a)语义解析(SP),其中使用节点和边嵌入生成逻辑形式(例如S-表达式、SPARQL查询等),然后在这些表示或调整语言模型的基础上进行推理,或(b)信息检索,该方法通过按顺序提取实体和关系来工作。 在这项工作中,我们评估了(LLMs)在回答涉及多个级的KG问题的能力。我们证明了,根据KG的大小和性质,我们需要不同的方法来提取和向LLM提供相关信息,因为每个LLM都具有固定的上下文窗口。我们在六个具有和没有例子特定子图的KG上评估我们的方法,并发现基于IR和SP的方法都可以被LLM采用,导致具有非常竞争力的性能。
https://arxiv.org/abs/2404.19234
Relational triple extraction is crucial work for the automatic construction of knowledge graphs. Existing methods only construct shallow representations from a token or token pair-level. However, previous works ignore local spatial dependencies of relational triples, resulting in a weakness of entity pair boundary detection. To tackle this problem, we propose a novel Region-based Table Filling method (RTF). We devise a novel region-based tagging scheme and bi-directional decoding strategy, which regard each relational triple as a region on the relation-specific table, and identifies triples by determining two endpoints of each region. We also introduce convolution to construct region-level table representations from a spatial perspective which makes triples easier to be captured. In addition, we share partial tagging scores among different relations to improve learning efficiency of relation classifier. Experimental results show that our method achieves state-of-the-art with better generalization capability on three variants of two widely used benchmark datasets.
关系三元提取对于知识图的自动构建至关重要。现有的方法仅从词或词对级别构建浅层次表示。然而,之前的 works忽略了关系三元组的局部空间依赖性,导致实体对边界检测的弱点。为了解决这个问题,我们提出了一个新颖的区域为基础的表填充方法(RTF)。我们设计了一个新颖的区域为基础的标签方案和双向解码策略,将每个关系三元组视为关系特定表中的一个区域,通过确定每个区域的两个端点来确定三元组。我们还引入卷积来从空间角度构建区域级别的表表示,使得三元组更容易被捕捉。此外,我们还在不同关系之间共享部分标签得分,以提高关系分类器的学习效率。实验结果表明,我们的方法在三个广泛使用基准数据集上实现了最先进的性能,且在扩展性方面表现出色。
https://arxiv.org/abs/2404.19154
Despite widespread applications of knowledge graphs (KGs) in various tasks such as question answering and intelligent conversational systems, existing KGs face two major challenges: information granularity and deficiency in timeliness. These hinder considerably the retrieval and analysis of in-context, fine-grained, and up-to-date knowledge from KGs, particularly in highly specialized themes (e.g., specialized scientific research) and rapidly evolving contexts (e.g., breaking news or disaster tracking). To tackle such challenges, we propose a theme-specific knowledge graph (i.e., ThemeKG), a KG constructed from a theme-specific corpus, and design an unsupervised framework for ThemeKG construction (named TKGCon). The framework takes raw theme-specific corpus and generates a high-quality KG that includes salient entities and relations under the theme. Specifically, we start with an entity ontology of the theme from Wikipedia, based on which we then generate candidate relations by Large Language Models (LLMs) to construct a relation ontology. To parse the documents from the theme corpus, we first map the extracted entity pairs to the ontology and retrieve the candidate relations. Finally, we incorporate the context and ontology to consolidate the relations for entity pairs. We observe that directly prompting GPT-4 for theme-specific KG leads to inaccurate entities (such as "two main types" as one entity in the query result) and unclear (such as "is", "has") or wrong relations (such as "have due to", "to start"). In contrast, by constructing the theme-specific KG step by step, our model outperforms GPT-4 and could consistently identify accurate entities and relations. Experimental results also show that our framework excels in evaluations compared with various KG construction baselines.
尽管知识图谱(KGs)在各种任务中的广泛应用,如问答和智能对话系统,现有KG面临两个主要挑战:信息粒度和时间不足。这些阻碍了从KGs中检索和分析上下文、细粒度和最新知识的能力,特别是在高度专业化的主题(例如,专业科学研究)和快速变化的环境(例如,新闻或灾害跟踪)中。为了应对这些挑战,我们提出了一个主题特定知识图(即 ThemeKG),一个基于主题特定语料库的知识图谱,并设计了用于 ThemeKG 构建的无监督框架(名为 TKGCon)。该框架从主题特定语料库中提取原始主题,然后通过大型语言模型(LLMs)生成候选关系,构建主题关系本体。为了解析主题语料库中的文档,我们首先将提取到的实体对映射到语料库,并检索候选关系。最后,我们将上下文和本体整合用于关系匹配。我们观察到,直接使用 GPT-4 生成主题特定 KG会导致不准确实体(例如查询结果中的“两个主要类型”作为一个实体),以及不清晰或错误的關係(例如“由於”或“开始于”)。相比之下,通过逐步构建主题特定 KG,我们的模型在比较各种 KG 建设基线方面表现出优异性能。实验结果还显示,我们的框架在各种 KG 建设基线上的评估中表现出色。
https://arxiv.org/abs/2404.19146
Knowledge graphs (KGs), which store an extensive number of relational facts (head, relation, tail), serve various applications. While many downstream tasks highly rely on the expressive modeling and predictive embedding of KGs, most of the current KG representation learning methods, where each entity is embedded as a vector in the Euclidean space and each relation is embedded as a transformation, follow an entity ranking protocol. On one hand, such an embedding design cannot capture many-to-many relations. On the other hand, in many retrieval cases, the users wish to get an exact set of answers without any ranking, especially when the results are expected to be precise, e.g., which genes cause an illness. Such scenarios are commonly referred to as "set retrieval". This work presents a pioneering study on the KG set retrieval problem. We show that the set retrieval highly depends on expressive modeling of many-to-many relations, and propose a new KG embedding model SpherE to address this problem. SpherE is based on rotational embedding methods, but each entity is embedded as a sphere instead of a vector. While inheriting the high interpretability of rotational-based models, our SpherE can more expressively model one-to-many, many-to-one, and many-to-many relations. Through extensive experiments, we show that our SpherE can well address the set retrieval problem while still having a good predictive ability to infer missing facts. The code is available at this https URL.
知识图(KGs)作为一种存储大量关系事实(头,关系,尾)的数据结构,具有各种应用价值。尽管许多下游任务高度依赖于KGs的表示建模和预测嵌入,但目前大多数KG表示学习方法,其中每个实体以欧氏空间中的向量表示,每个关系以变换表示,都遵循实体排序协议。一方面,这种嵌入设计无法捕捉许多对多关系。另一方面,在许多检索案例中,用户希望获得一个无排名的准确集合答案,尤其是在结果预计精确的情况下,例如哪些基因导致疾病。这种情况通常被称为“集检索”。 本文在KG集检索问题上进行了一项开创性的研究。我们证明了集检索高度依赖于多对多关系的表示建模,并提出了一个新的KG嵌入模型SpherE来解决这个问题。SpherE基于旋转嵌入方法,但每个实体都被嵌入为一个球体而不是向量。虽然继承了旋转模型的高可解释性,但我们的SpherE可以更富有表现力地建模一对一、一对多和多对多关系。通过大量实验,我们证明了我们的SpherE可以在解决集检索问题的同时,仍具有推断缺失事实的良好预测能力。代码可在此处访问:https://www.acm.org/dl/d/2222216
https://arxiv.org/abs/2404.19130