To deduce new facts on a knowledge graph (KG), a link predictor learns from the graph structure and collects local evidence to find the answer to a given query. However, existing methods suffer from a severe scalability problem due to the utilization of the whole KG for prediction, which hinders their promise on large scale KGs and cannot be directly addressed by vanilla sampling methods. In this work, we propose the one-shot-subgraph link prediction to achieve efficient and adaptive prediction. The design principle is that, instead of directly acting on the whole KG, the prediction procedure is decoupled into two steps, i.e., (i) extracting only one subgraph according to the query and (ii) predicting on this single, query dependent subgraph. We reveal that the non-parametric and computation-efficient heuristics Personalized PageRank (PPR) can effectively identify the potential answers and supporting evidence. With efficient subgraph-based prediction, we further introduce the automated searching of the optimal configurations in both data and model spaces. Empirically, we achieve promoted efficiency and leading performances on five large-scale benchmarks. The code is publicly available at: this https URL.
通过从知识图谱(KG)中推断新事实,链接预测器从图结构中学习并收集局部证据来寻找给定查询的答案。然而,由于利用整个KG进行预测,现有方法存在严重的可扩展性问题,这阻碍了它们在大型KG上的承诺,并且不能通过普通采样方法直接解决。在本文中,我们提出了一击子图链接预测,以实现高效和自适应的预测。设计原则是,预测过程被分解成两个步骤,即(i)根据查询提取只有一个子图,(ii)在这个单个、查询相关的子图上进行预测。我们发现,非参数且计算效率高的启发式个人化页面排名(PPR)可以有效地识别潜在答案和支持证据。通过高效基于子图的预测,我们进一步引入了在数据和模型空间中自动搜索最优配置的自动化过程。经验证,我们在五个大型KG基准测试中实现了提升的效率和卓越的性能。代码公开可用,在此https URL。
https://arxiv.org/abs/2403.10231
In recent advancements within the domain of Large Language Models (LLMs), there has been a notable emergence of agents capable of addressing Robotic Process Automation (RPA) challenges through enhanced cognitive capabilities and sophisticated reasoning. This development heralds a new era of scalability and human-like adaptability in goal attainment. In this context, we introduce AUTONODE (Autonomous User-interface Transformation through Online Neuro-graphic Operations and Deep Exploration). AUTONODE employs advanced neuro-graphical techniques to facilitate autonomous navigation and task execution on web interfaces, thereby obviating the necessity for predefined scripts or manual intervention. Our engine empowers agents to comprehend and implement complex workflows, adapting to dynamic web environments with unparalleled efficiency. Our methodology synergizes cognitive functionalities with robotic automation, endowing AUTONODE with the ability to learn from experience. We have integrated an exploratory module, DoRA (Discovery and mapping Operation for graph Retrieval Agent), which is instrumental in constructing a knowledge graph that the engine utilizes to optimize its actions and achieve objectives with minimal supervision. The versatility and efficacy of AUTONODE are demonstrated through a series of experiments, highlighting its proficiency in managing a diverse array of web-based tasks, ranging from data extraction to transaction processing.
在大型语言模型(LLMs)领域最近的研究进展中,出现了一些能够通过增强认知能力和复杂的推理能力来应对机器人流程自动化(RPA)挑战的智能体。这一发展预示着在实现目标的过程中将进入一个可扩展和具有人类相似适应性的新时代。在这个背景下,我们介绍了一个名为AUTONODE(通过在线神经图网络操作和深度探索实现自主用户界面转换)的系统。AUTONODE采用先进的神经图网络技术来促进自主导航和任务执行在网页界面上,从而消除了需要预定义脚本或手动干预的必要性。我们的引擎使智能体能够理解并实施复杂的任务流程,适应于动态的网页环境,效率无与伦比。我们的方法论将认知功能与机器人自动化相结合,使AUTONODE具有从经验中学习的能力。我们引入了一个探索模块,DoRA(用于构建知识图的发现和映射操作),该模块对于构建引擎使用的知识图至关重要。通过一系列实验,我们展示了AUTONODE的多样性和有效性,涵盖了从数据提取到交易处理的各类网页任务。
https://arxiv.org/abs/2403.10171
This paper introduces a novel Functional Graph Convolutional Network (funGCN) framework that combines Functional Data Analysis and Graph Convolutional Networks to address the complexities of multi-task and multi-modal learning in digital health and longitudinal studies. With the growing importance of health solutions to improve health care and social support, ensure healthy lives, and promote well-being at all ages, funGCN offers a unified approach to handle multivariate longitudinal data for multiple entities and ensures interpretability even with small sample sizes. Key innovations include task-specific embedding components that manage different data types, the ability to perform classification, regression, and forecasting, and the creation of a knowledge graph for insightful data interpretation. The efficacy of funGCN is validated through simulation experiments and a real-data application.
本文提出了一种新颖的函数图卷积网络(funGCN)框架,将功能数据分析和图卷积网络相结合,以解决数字健康和纵向研究中的多任务和多模态学习复杂性。随着医疗解决方案在改善医疗保健和社会支持、确保健康生活和促进各年龄段的健康益处方面的重要性不断增加,funGCN为多个实体处理多维纵向数据提供了一个统一的解决方案,并且即使样本量较小,也能确保可解释性。关键创新包括特定任务嵌入组件,用于管理不同数据类型,进行分类、回归和预测的能力,以及创建了用于洞察数据解释的知识图。funGCN的有效性通过仿真实验和实际应用得到了验证。
https://arxiv.org/abs/2403.10158
Knowledge graphs contain informative factual knowledge but are considered incomplete. To answer complex queries under incomplete knowledge, learning-based Complex Query Answering (CQA) models are proposed to directly learn from the query-answer samples to avoid the direct traversal of incomplete graph data. Existing works formulate the training of complex query answering models as multi-task learning and require a large number of training samples. In this work, we explore the compositional structure of complex queries and argue that the different logical operator types, rather than the different complex query types, are the key to improving generalizability. Accordingly, we propose a meta-learning algorithm to learn the meta-operators with limited data and adapt them to different instances of operators under various complex queries. Empirical results show that learning meta-operators is more effective than learning original CQA or meta-CQA models.
知识图包含有益的事实性知识,但被认为是不完整的。为了解决在不完整知识下回答复杂查询的问题,学习模型的复杂查询回答(CQA)模型被提出,可以直接从查询-答案样本中学习,以避免对不完整图形数据的直接遍历。现有工作将复杂查询回答模型的训练视为多任务学习,需要大量的训练样本。在本文中,我们探讨了复杂查询的组合结构,并认为不同逻辑操作符类型,而不是不同复杂查询类型,是提高可扩展性的关键。因此,我们提出了一种元学习算法,以在有限数据的情况下学习元操作符,并将它们适应各种复杂查询的实例。实证结果表明,学习元操作符比学习原始CQA或元CQA模型更有效。
https://arxiv.org/abs/2403.10110
While the introduction of contrastive learning frameworks in sentence representation learning has significantly contributed to advancements in the field, it still remains unclear whether state-of-the-art sentence embeddings can capture the fine-grained semantics of sentences, particularly when conditioned on specific perspectives. In this paper, we introduce Hyper-CL, an efficient methodology that integrates hypernetworks with contrastive learning to compute conditioned sentence representations. In our proposed approach, the hypernetwork is responsible for transforming pre-computed condition embeddings into corresponding projection layers. This enables the same sentence embeddings to be projected differently according to various conditions. Evaluation on two representative conditioning benchmarks, namely conditional semantic text similarity and knowledge graph completion, demonstrates that Hyper-CL is effective in flexibly conditioning sentence representations, showcasing its computational efficiency at the same time. We also provide a comprehensive analysis of the inner workings of our approach, leading to a better interpretation of its mechanisms.
尽管在句子表示学习领域引入对比性学习框架已经显著推动了进步,但仍然不清楚最先进的句子嵌入是否能够捕捉到句子中微妙的语义,特别是当特定视角下时。在本文中,我们引入了Hyper-CL,一种将超网络与对比学习相结合的有效方法,用于计算有条件句子表示。在我们的方法中,超网络负责将预计算的有条件嵌入转换为相应的投影层。这使得根据各种条件投影相同的句子嵌入。在两个具有代表性的 conditioning 基准(即条件语义文本相似度和知识图谱完成)上的评估表明,Hyper-CL 有效地调节了句子表示,同时展示了其在计算效率方面的优势。我们还对我们的方法进行了全面的分析,从而更好地解释了其机制。
https://arxiv.org/abs/2403.09490
Large Language Models (LLMs) have shown potential in reasoning over structured environments, e.g., knowledge graph and table. Such tasks typically require multi-hop reasoning, i.e., match natural language utterance with instances in the environment. Previous methods leverage LLMs to incrementally build a reasoning path, where the LLMs either invoke tools or pick up schemas by step-by-step interacting with the environment. We propose Reasoning-Path-Editing (Readi), a novel framework where LLMs can efficiently and faithfully reason over structured environments. In Readi, LLMs initially generate a reasoning path given a query, and edit the path only when necessary. We instantiate the path on structured environments and provide feedback to edit the path if anything goes wrong. Experimental results on three KGQA datasets and two TableQA datasets show the effectiveness of Readi, significantly surpassing all LLM-based methods (by 9.1% on WebQSP, 12.4% on MQA-3H and 10.9% on WTQ), comparable with state-of-the-art fine-tuned methods (67% on CWQ and 74.7% on WebQSP) and substantially boosting the vanilla LLMs (by 14.9% on CWQ). Our code will be available upon publication.
大语言模型(LLMs)在结构化环境中表现出推理潜力,例如知识图谱和表格。这类任务通常需要多级推理,即自然语言句子与环境中实例的匹配。之前的方法利用LLMs逐步构建推理路径,其中LLMs通过逐步与环境的交互选择或调用工具来获取模式。我们提出了一种名为“推理路径编辑”(Readi)的新框架,LLMs可以在结构化环境中高效且准确地进行推理。在Readi中,LLMs首先根据查询生成推理路径,并在需要时进行编辑。我们在结构化环境中实例化路径,并在路径出现问题时提供反馈进行编辑。在三个KGQA数据集和两个TableQA数据集上的实验结果表明,Readi的有效性超过了所有LLM基方法(在WebQSP上提高了9.1%,在MQA-3H上提高了12.4%,在WTQ上提高了10.9%),与最先进的微调方法(在CWQ上提高了67%,在WebQSP上提高了74.7%)相当,显著提高了原生的LLM(在CWQ上提高了14.9%,在WebQSP上提高了18.2%)。我们的代码将在发表后公开发布。
https://arxiv.org/abs/2403.08593
Federated learning (FL) promotes the development and application of artificial intelligence technologies by enabling model sharing and collaboration while safeguarding data privacy. Knowledge graph (KG) embedding representation provides a foundation for knowledge reasoning and applications by mapping entities and relations into vector space. Federated KG embedding enables the utilization of knowledge from diverse client sources while safeguarding the privacy of local data. However, due to demands such as privacy protection and the need to adapt to dynamic data changes, investigations into machine unlearning (MU) have been sparked. However, it is challenging to maintain the performance of KG embedding models while forgetting the influence of specific forgotten data on the model. In this paper, we propose FedDM, a novel framework tailored for machine unlearning in federated knowledge graphs. Leveraging diffusion models, we generate noisy data to sensibly mitigate the influence of specific knowledge on FL models while preserving the overall performance concerning the remaining data. We conduct experimental evaluations on benchmark datasets to assess the efficacy of the proposed model. Extensive experiments demonstrate that FedDM yields promising results in knowledge forgetting.
联邦学习(FL)通过促进模型共享和协作,同时保护数据隐私,推动人工智能技术的发展和应用。知识图(KG)嵌入表示为知识推理和应用提供了一个基础,将实体和关系映射到向量空间。联邦KG嵌入使来自不同客户端的知识得以利用,同时保护本地数据的隐私。然而,由于隐私保护和适应动态数据变化的需求,机器学习(ML)领域的研究也引发了兴趣。然而,忘记具体遗忘数据对模型性能的影响使得维持KG嵌入模型的性能具有挑战性。在本文中,我们提出了FedDM,一种专门为联邦知识图中的机器学习而设计的框架。利用扩散模型,我们生成噪声数据,有效地抵消了特定知识对FL模型的影响,同时保留剩余数据的整体性能。我们对基准数据集进行实验评估,以评估所提出的模型的有效性。大量实验证明,FedDM在知识遗忘方面具有鼓舞人心的结果。
https://arxiv.org/abs/2403.08554
The conventional process of building Ontologies and Knowledge Graphs (KGs) heavily relies on human domain experts to define entities and relationship types, establish hierarchies, maintain relevance to the domain, fill the ABox (or populate with instances), and ensure data quality (including amongst others accuracy and completeness). On the other hand, Large Language Models (LLMs) have recently gained popularity for their ability to understand and generate human-like natural language, offering promising ways to automate aspects of this process. This work explores the (semi-)automatic construction of KGs facilitated by open-source LLMs. Our pipeline involves formulating competency questions (CQs), developing an ontology (TBox) based on these CQs, constructing KGs using the developed ontology, and evaluating the resultant KG with minimal to no involvement of human experts. We showcase the feasibility of our semi-automated pipeline by creating a KG on deep learning methodologies by exploiting scholarly publications. To evaluate the answers generated via Retrieval-Augmented-Generation (RAG) as well as the KG concepts automatically extracted using LLMs, we design a judge LLM, which rates the generated content based on ground truth. Our findings suggest that employing LLMs could potentially reduce the human effort involved in the construction of KGs, although a human-in-the-loop approach is recommended to evaluate automatically generated KGs.
传统的建设语义网和知识图(KGs)的过程很大程度上依赖于人类领域专家来定义实体和关系类型,建立层次结构,保持领域相关性,填充ABox(或实例),并确保数据质量(包括准确性、完整性等)。另一方面,大型语言模型(LLMs)最近因它们能够理解并生成类似人类自然语言而受到欢迎,为自动化这个过程提供了有前途的方法。这项工作探讨了通过开源LLM自动构建知识图(KGs)的可能性。我们的工作流程包括制定能力问题(CQs),基于这些CQs构建语义网(TBox),使用所开发的语义网构建KG,并对由此产生的KG进行评估,最小限度地不涉及人类专家。我们通过利用学术出版物来创建深度学习方法论的KG,展示了我们的半自动化的流程的可行性。为了评估通过检索增强生成(RAG)生成的答案以及使用LLM自动提取的KG概念,我们设计了一个评分LLM,根据真实值对生成内容进行评分。我们的研究结果表明,使用LLM可能有助于减少构建KGs的人力投入,尽管建议使用人类在循环中评估自动生成的KG。
https://arxiv.org/abs/2403.08345
In the midst of widespread misinformation and disinformation through social media and the proliferation of AI-generated texts, it has become increasingly difficult for people to validate and trust information they encounter. Many fact-checking approaches and tools have been developed, but they often lack appropriate explainability or granularity to be useful in various contexts. A text validation method that is easy to use, accessible, and can perform fine-grained evidence attribution has become crucial. More importantly, building user trust in such a method requires presenting the rationale behind each prediction, as research shows this significantly influences people's belief in automated systems. It is also paramount to localize and bring users' attention to the specific problematic content, instead of providing simple blanket labels. In this paper, we present $\textit{ClaimVer, a human-centric framework}$ tailored to meet users' informational and verification needs by generating rich annotations and thereby reducing cognitive load. Designed to deliver comprehensive evaluations of texts, it highlights each claim, verifies it against a trusted knowledge graph (KG), presents the evidence, and provides succinct, clear explanations for each claim prediction. Finally, our framework introduces an attribution score, enhancing applicability across a wide range of downstream tasks.
在社交媒体和AI生成的文本广泛传播不准确和误导信息的大背景下,人们越来越难以验证和信任他们所遇到的信息。已经开发了许多验证方法和工具,但它们通常缺乏适当的可解释性或粒度,使其在各种上下文中没有用处。一种易于使用、可访问且能进行细粒度证据归因的文本验证方法变得至关重要。更重要的是,在这样一种方法中建立用户对其的信任,正如研究显示,这显著影响着人们相信自动系统。此外,将用户集中在具体问题内容上,而不是提供简单的覆盖标签,也非常重要。在本文中,我们提出了ClaimVer,一种以满足用户信息验证和确认需求的以人为本框架,通过生成丰富的注释来降低认知负担。旨在全面评估文本,它突出每个主张,将其与可信知识图(KG)进行验证,提供每个主张预测的清晰、简洁的解释。最后,我们的框架引入了归因分数,提高了在广泛的下游任务中的适用性。
https://arxiv.org/abs/2403.09724
Histopathological whole slide images (WSIs) classification has become a foundation task in medical microscopic imaging processing. Prevailing approaches involve learning WSIs as instance-bag representations, emphasizing significant instances but struggling to capture the interactions between instances. Additionally, conventional graph representation methods utilize explicit spatial positions to construct topological structures but restrict the flexible interaction capabilities between instances at arbitrary locations, particularly when spatially distant. In response, we propose a novel dynamic graph representation algorithm that conceptualizes WSIs as a form of the knowledge graph structure. Specifically, we dynamically construct neighbors and directed edge embeddings based on the head and tail relationships between instances. Then, we devise a knowledge-aware attention mechanism that can update the head node features by learning the joint attention score of each neighbor and edge. Finally, we obtain a graph-level embedding through the global pooling process of the updated head, serving as an implicit representation for the WSI classification. Our end-to-end graph representation learning approach has outperformed the state-of-the-art WSI analysis methods on three TCGA benchmark datasets and in-house test sets. Our code is available at this https URL.
病理学全切片图像(WSIs)分类已经成为医学显微镜图像处理的基础任务。现有的方法将WSIs学习为实例袋表示,强调显著实例,但很难捕捉实例之间的相互作用。此外,传统的图表示方法利用明确的空间位置来构建拓扑结构,但限制了实例在任意位置的灵活交互功能,特别是在空间距离较远的情况下。为了应对这个问题,我们提出了一个新颖的动态图表示算法,将WSIs视为知识图的一种形式。具体来说,我们根据实例的头部和尾部关系动态构建邻居和指向边缘嵌入。然后,我们设计了一个知识感知注意机制,可以通过学习每个邻居和边缘的联合注意分数来更新头节点特征。最后,我们通过更新更新的头节点的全局池化过程获得图级别嵌入,作为WSI分类的隐含表示。我们的端到端图表示学习方法在三个TCGA基准数据集和内部测试集上超过了最先进的WSI分析方法的表现。我们的代码可在此处访问:https://url.com/。
https://arxiv.org/abs/2403.07719
Event commonsense reasoning requires the ability to reason about the relationship between events, as well as infer implicit context underlying that relationship. However, data scarcity makes it challenging for language models to learn to generate commonsense inferences for contexts and questions involving interactions between complex events. To address this demand, we present COM2 (COMplex COMmonsense), a new dataset created by sampling multi-hop logical queries (e.g., the joint effect or cause of both event A and B, or the effect of the effect of event C) from an existing commonsense knowledge graph (CSKG), and verbalizing them using handcrafted rules and large language models into multiple-choice and text generation questions. Our experiments show that language models trained on COM2 exhibit significant improvements in complex reasoning ability, resulting in enhanced zero-shot performance in both in-domain and out-of-domain tasks for question answering and generative commonsense reasoning, without expensive human annotations.
事件常识推理需要能够推理事件之间的关系,以及推断隐含在这些关系背后的上下文。然而,数据稀缺使得语言模型难以学会为涉及复杂事件交互的上下文和问题生成常识推理。为了解决这个需求,我们提出了COM2(复杂事件常识),一个新的数据集,通过从现有常识知识图谱(CSKG)中采样多级逻辑查询(例如,事件A和事件B的共同影响或事件C的影响)创建,并使用手工规则和大型语言模型进行阐述,生成多个选择题和文本生成问题。我们的实验结果表明,训练在COM2上的语言模型在复杂推理能力上表现出显著的提高,从而在问答和生成常识推理方面实现了在领域内和领域外任务的显著提升,而无需昂贵的 human 注释。
https://arxiv.org/abs/2403.07398
Currently, little research has been done on knowledge editing for Large Vision-Language Models (LVLMs). Editing LVLMs faces the challenge of effectively integrating diverse modalities (image and text) while ensuring coherent and contextually relevant modifications. An existing benchmark has three metrics (Reliability, Locality and Generality) to measure knowledge editing for LVLMs. However, the benchmark falls short in the quality of generated images used in evaluation and cannot assess whether models effectively utilize edited knowledge in relation to the associated content. We adopt different data collection methods to construct a new benchmark, $\textbf{KEBench}$, and extend new metric (Portability) for a comprehensive evaluation. Leveraging a multimodal knowledge graph, our image data exhibits clear directionality towards entities. This directional aspect can be further utilized to extract entity-related knowledge and form editing data. We conducted experiments of different editing methods on five LVLMs, and thoroughly analyze how these methods impact the models. The results reveal strengths and deficiencies of these methods and, hopefully, provide insights into potential avenues for future research.
目前,对于大型视觉语言模型(LVLMs),知识编辑的研究还很少。编辑LVLMs面临着将多种模态(图像和文本)有效地整合起来,同时确保连贯性和相关性的修改。现有的基准(Reliability, Locality和Generality)只能测量LVLMs的知识编辑质量。然而,该基准在评估中使用的生成图像的质量上存在不足,并且无法评估模型是否有效利用编辑知识与相关内容之间的关系。我们采用不同的数据收集方法构建了一个新的基准点(KEBench),并为综合评估引入了一个新的指标(可移植性)。利用多模态知识图谱,我们的图像数据表现出明显的面向实体的方向性。这种方向性可以进一步用于提取实体相关的知识并形成编辑数据。我们在五种LVLMs上进行了不同编辑方法的实验,并对这些方法对模型的影响进行了详细分析。结果揭示了这些方法的优缺点,有望为未来的研究提供有价值的启示。
https://arxiv.org/abs/2403.07350
The task of predicting multiple links within knowledge graphs (KGs) stands as a challenge in the field of knowledge graph analysis, a challenge increasingly resolvable due to advancements in natural language processing (NLP) and KG embedding techniques. This paper introduces a novel methodology, the Knowledge Graph Large Language Model Framework (KG-LLM), which leverages pivotal NLP paradigms, including chain-of-thought (CoT) prompting and in-context learning (ICL), to enhance multi-hop link prediction in KGs. By converting the KG to a CoT prompt, our framework is designed to discern and learn the latent representations of entities and their interrelations. To show the efficacy of the KG-LLM Framework, we fine-tune three leading Large Language Models (LLMs) within this framework, employing both non-ICL and ICL tasks for a comprehensive evaluation. Further, we explore the framework's potential to provide LLMs with zero-shot capabilities for handling previously unseen prompts. Our experimental findings discover that integrating ICL and CoT not only augments the performance of our approach but also significantly boosts the models' generalization capacity, thereby ensuring more precise predictions in unfamiliar scenarios.
在知识图谱(KG)领域预测多个链接是一个挑战,由于自然语言处理(NLP)和KG嵌入技术的进步,这个挑战正在逐渐得到解决。本文介绍了一种新方法,知识图谱大语言模型框架(KG-LLM),它利用关键的NLP范式,包括连锁思考(CoT)提示和上下文学习(ICL),来增强KG中的多级链接预测。通过将KG转换为CoT提示,我们的框架旨在分辨和 learn 实体及其关系的潜在表示。为了展示KG-LLM框架的有效性,我们在该框架中微调了三个领先的大语言模型(LLMs),同时使用非ICL和ICL任务进行全面的评估。此外,我们探讨了该框架为LLM提供零散化能力处理以前未见过的提示的可能性。我们的实验发现,整合ICL和CoT不仅增强了我们方法的表现,而且显著提高了模型的泛化能力,从而确保在陌生场景中进行更精确的预测。
https://arxiv.org/abs/2403.07311
The use of generative AI to create text descriptions from graphs has mostly focused on knowledge graphs, which connect concepts using facts. In this work we explore the capability of large pretrained language models to generate text from causal graphs, where salient concepts are represented as nodes and causality is represented via directed, typed edges. The causal reasoning encoded in these graphs can support applications as diverse as healthcare or marketing. Using two publicly available causal graph datasets, we empirically investigate the performance of four GPT-3 models under various settings. Our results indicate that while causal text descriptions improve with training data, compared to fact-based graphs, they are harder to generate under zero-shot settings. Results further suggest that users of generative AI can deploy future applications faster since similar performances are obtained when training a model with only a few examples as compared to fine-tuning via a large curated dataset.
翻译:将生成式AI用于从图形中创建文本描述的主要集中在知识图谱上,通过事实连接概念。在这项工作中,我们探讨了大型预训练语言模型在因果图上生成文本的能力,其中显著的概念用节点表示,因果关系通过有向、带标签的边表示。这些图形中的因果推理可以支持各种应用,如医疗保健或市场营销。使用两个公开可用的因果图数据集,我们通过不同设置下的实验,研究了四种GPT-3模型的性能。我们的结果表明,随着训练数据的增加,因果文本描述的改善,与基于事实的图形相比,它们在零散设置下生成文本较为困难。结果进一步表明,使用生成式AI的用户可以更快地部署未来的应用,因为与通过大型编辑数据集训练模型相比,获得类似性能时只需要几个示例。
https://arxiv.org/abs/2403.07118
Knowledge graph embeddings (KGEs) were originally developed to infer true but missing facts in incomplete knowledge repositories. In this paper, we link knowledge graph completion and counterfactual reasoning via our new task CFKGR. We model the original world state as a knowledge graph, hypothetical scenarios as edges added to the graph, and plausible changes to the graph as inferences from logical rules. We create corresponding benchmark datasets, which contain diverse hypothetical scenarios with plausible changes to the original knowledge graph and facts that should be retained. We develop COULDD, a general method for adapting existing knowledge graph embeddings given a hypothetical premise, and evaluate it on our benchmark. Our results indicate that KGEs learn patterns in the graph without explicit training. We further observe that KGEs adapted with COULDD solidly detect plausible counterfactual changes to the graph that follow these patterns. An evaluation on human-annotated data reveals that KGEs adapted with COULDD are mostly unable to recognize changes to the graph that do not follow learned inference rules. In contrast, ChatGPT mostly outperforms KGEs in detecting plausible changes to the graph but has poor knowledge retention. In summary, CFKGR connects two previously distinct areas, namely KG completion and counterfactual reasoning.
知识图嵌入(KGEs)最初是为了推断完整知识库中存在的真实但缺失的事实。在本文中,我们通过我们的新任务CFKGR将知识图的完成和反事实推理连接起来。我们将原始世界状态建模为知识图,假设情景作为图中的边,推测对知识图的更改相当于从逻辑规则得出的推理。我们创建了相应的基准数据集,其中包含具有对原始知识图的合理更改和应该保留的事实的多样假设情景。我们开发了COULDD,一种适应现有知识图嵌入的通用方法,并将其应用于我们的基准。我们的结果表明,KGEs在没有显式训练的情况下学会了图中的模式。我们进一步观察到,使用COULDD,KGEs能够稳健地检测到符合这些模式的可预测的反事实更改。在人类标注数据上的评估显示,使用COULDD适应的KGEs大多无法识别不符合学习推理规则的图形更改。相比之下,ChatGPT在检测图形反事实更改方面表现出色,但其知识保留能力较弱。总之,CFKGR将两个之前独立的部分连接起来,即知识图的完成和反事实推理。
https://arxiv.org/abs/2403.06936
The advancement of Multi-modal Pre-training highlights the necessity for a robust Multi-Modal Knowledge Graph (MMKG) representation learning framework. This framework is crucial for integrating structured knowledge into multi-modal Large Language Models (LLMs) at scale, aiming to alleviate issues like knowledge misconceptions and multi-modal hallucinations. In this work, to evaluate models' ability to accurately embed entities within MMKGs, we focus on two widely researched tasks: Multi-modal Knowledge Graph Completion (MKGC) and Multi-modal Entity Alignment (MMEA). Building on this foundation, we propose a novel SNAG method that utilizes a Transformer-based architecture equipped with modality-level noise masking for the robust integration of multi-modal entity features in KGs. By incorporating specific training objectives for both MKGC and MMEA, our approach achieves SOTA performance across a total of ten datasets (three for MKGC and seven for MEMA), demonstrating its robustness and versatility. Besides, SNAG can not only function as a standalone model but also enhance other existing methods, providing stable performance improvements. Our code and data are available at: this https URL.
多模态预训练的发展强调了需要一个健壮的多模态知识图(MMKG)表示学习框架。这个框架对于将结构化知识整合到大规模的多模态大型语言模型(LLM)中至关重要,旨在解决诸如知识误解和多模态幻觉等问题。在这项工作中,为了评估模型在MMKG中准确嵌入实体的能力,我们关注两个研究的热门任务:多模态知识图完成(MKGC)和多模态实体对齐(MMEA)。在此基础上,我们提出了一个新颖的SNAG方法,该方法采用基于Transformer的架构,并配备了模态级别噪声掩码,以实现对多模态实体特征的稳健整合。通过为MKGC和MMEA specific训练目标,我们的方法在总共10个数据集(其中三个为MKGC,七个为MMEA)上实现了SOTA性能,证明了其稳健性和多样性。此外,SNAG不仅可以作为一个独立的模型,还可以增强其他现有方法,提供稳定的性能提升。我们的代码和数据可在此处访问:https://this URL。
https://arxiv.org/abs/2403.06832
With appropriate data selection and training techniques, Large Language Models (LLMs) have demonstrated exceptional success in various medical examinations and multiple-choice questions. However, the application of LLMs in medical dialogue generation-a task more closely aligned with actual medical practice-has been less explored. This gap is attributed to the insufficient medical knowledge of LLMs, which leads to inaccuracies and hallucinated information in the generated medical responses. In this work, we introduce the Medical dialogue with Knowledge enhancement and clinical Pathway encoding (MedKP) framework, which integrates an external knowledge enhancement module through a medical knowledge graph and an internal clinical pathway encoding via medical entities and physician actions. Evaluated with comprehensive metrics, our experiments on two large-scale, real-world online medical consultation datasets (MedDG and KaMed) demonstrate that MedKP surpasses multiple baselines and mitigates the incidence of hallucinations, achieving a new state-of-the-art. Extensive ablation studies further reveal the effectiveness of each component of MedKP. This enhancement advances the development of reliable, automated medical consultation responses using LLMs, thereby broadening the potential accessibility of precise and real-time medical assistance.
凭借适当的數據選擇和訓練技術,大型語言模型(LLMs)在各種醫學檢驗和多选题中展現了非凡的成功。然而,在將LLMs應用於醫學對話生成-一個更接近實際醫療實踐的任務-方面,探討還不夠深入。這個缺口歸因於LLMs的醫學知識不足,導致生成的醫療回應不準確和出現幻影信息。在這個工作中,我們介紹了 Medical dialogue with Knowledge enhancement and clinical Pathway encoding (MedKP) 框架,該框架通過醫學知識圖形和醫療實體以及醫生的行動來實現外部知識增強,並通過醫學知識圖形和醫學實體來實現內部的臨床 pathway 編碼。使用全面的指標進行評估,我們在兩個大型、現實世界的在線醫療咨詢數據集(MedDG 和 KaMed)上的實驗證明了 MedKP 超越了多個基線,減少了幻影的發生,實現了最新的狀態。進一步的消減研究揭示了每個 MedKP 組件的有效性。這種增強進一步推動了使用LLM來開發可靠、自動化的醫療咨詢響應,從而擴展了對精确和實時醫療援助的精確可及性。
https://arxiv.org/abs/2403.06611
Knowledge-based question answering (KBQA) is a key task in NLP research, and also an approach to access the web data and knowledge, which requires exploiting knowledge graphs (KGs) for reasoning. In the literature, one promising solution for KBQA is to incorporate the pretrained language model (LM) with KGs by generating KG-centered pretraining corpus, which has shown its superiority. However, these methods often depend on specific techniques and resources to work, which may not always be available and restrict its application. Moreover, existing methods focus more on improving language understanding with KGs, while neglect the more important human-like complex reasoning. To this end, in this paper, we propose a general Knowledge-Injected Curriculum Pretraining framework (KICP) to achieve comprehensive KG learning and exploitation for KBQA tasks, which is composed of knowledge injection (KI), knowledge adaptation (KA) and curriculum reasoning (CR). Specifically, the KI module first injects knowledge into the LM by generating KG-centered pretraining corpus, and generalizes the process into three key steps that could work with different implementations for flexible application. Next, the KA module learns knowledge from the generated corpus with LM equipped with an adapter as well as keeps its original natural language understanding ability to reduce the negative impacts of the difference between the generated and natural corpus. Last, to enable the LM with complex reasoning, the CR module follows human reasoning patterns to construct three corpora with increasing difficulties of reasoning, and further trains the LM from easy to hard in a curriculum manner. We provide an implementation of the general framework, and evaluate the proposed KICP on four real-word datasets. The results demonstrate that our framework can achieve higher performances.
知识图谱(KG)是自然语言处理(NLP)研究的一个关键任务,也是访问网络数据和知识的途径,这需要利用知识图谱(KG)进行推理。在文献中,一个有前景的KBQA解决方案是将预训练语言模型(LM)与KG结合,通过生成以KG为中心的预训练语料库来实现,这已经展示了其优越性。然而,这些方法通常依赖于特定的技术和资源才能工作,这可能并不总是可用的,从而限制了其应用范围。此外,现有的方法更多地关注通过KG提高语言理解,而忽略了更重要的类人推理。为此,在本文中,我们提出了一个通用的知识注入课程预训练框架(KICP),以实现KG学习的全面性和对KBQA任务的充分利用,该框架由知识注入(KI)、知识适应(KA)和课程推理(CR)组成。具体来说,KI模块首先通过生成以KG为中心的预训练语料库将知识注入到LM中,并将其扩展到三种可能适用于不同实现方法的关键步骤。接下来,KA模块使用配备有 adapter 的LM从生成的语料库中学习知识,并保留其原有的自然语言理解能力,以减少生成的自然语料库与自然语料库之间的差异对负面的影响。最后,为了实现具有复杂推理能力的LM,CR模块遵循人类推理模式,从容易到困难构建三个语料库,并进一步以课程方式训练LM。我们提供了该通用框架的实现,并在四个真实世界数据集上进行了评估。结果表明,我们的框架可以达到更高的性能。
https://arxiv.org/abs/2403.09712
Large Language Models (LLMs) have significantly advanced healthcare innovation on generation capabilities. However, their application in real clinical settings is challenging due to potential deviations from medical facts and inherent biases. In this work, we develop an augmented LLM framework, KG-Rank, which leverages a medical knowledge graph (KG) with ranking and re-ranking techniques, aiming to improve free-text question-answering (QA) in the medical domain. Specifically, upon receiving a question, we initially retrieve triplets from a medical KG to gather factual information. Subsequently, we innovatively apply ranking methods to refine the ordering of these triplets, aiming to yield more precise answers. To the best of our knowledge, KG-Rank is the first application of ranking models combined with KG in medical QA specifically for generating long answers. Evaluation of four selected medical QA datasets shows that KG-Rank achieves an improvement of over 18% in the ROUGE-L score. Moreover, we extend KG-Rank to open domains, where it realizes a 14% improvement in ROUGE-L, showing the effectiveness and potential of KG-Rank.
大语言模型(LLMs)在生成能力方面显著推动了医疗技术的创新。然而,它们在现实临床环境中的应用具有挑战性,因为可能存在与医学事实不符的潜在偏差。在这项工作中,我们开发了一个增强的LLM框架,KG-Rank,利用医疗知识图(KG)的排名和重新排名技术,旨在提高医疗领域的自由文本问题解答(QA)能力。具体来说,在接收到一个问题后,我们首先从医疗KG中检索三元组以收集事实信息。随后,我们创新地应用排名方法来优化这些三元组的排序,旨在产生更精确的答案。据我们所知,KG-Rank是针对生成长篇答案的首个应用排名模型与KG的结合。对四个选定的医疗QA数据集进行评估,KG-Rank在ROUGE-L得分方面的改进超过了18%。此外,我们将KG-Rank扩展到开放领域,实现了超过14%的ROUGE-L改进,展示了KG-Rank的有效性和潜在。
https://arxiv.org/abs/2403.05881
Despite advancements in on-topic dialogue systems, effectively managing topic shifts within dialogues remains a persistent challenge, largely attributed to the limited availability of training datasets. To address this issue, we propose Multi-Passage to Dialogue (MP2D), a data generation framework that automatically creates conversational question-answering datasets with natural topic transitions. By leveraging the relationships between entities in a knowledge graph, MP2D maps the flow of topics within a dialogue, effectively mirroring the dynamics of human conversation. It retrieves relevant passages corresponding to the topics and transforms them into dialogues through the passage-to-dialogue method. Through quantitative and qualitative experiments, we demonstrate MP2D's efficacy in generating dialogue with natural topic shifts. Furthermore, this study introduces a novel benchmark for topic shift dialogues, TS-WikiDialog. Utilizing the dataset, we demonstrate that even Large Language Models (LLMs) struggle to handle topic shifts in dialogue effectively, and we showcase the performance improvements of models trained on datasets generated by MP2D across diverse topic shift dialogue tasks.
尽管在主题对话系统方面取得了进步,但在对话中有效地管理主题转移仍然是一个持续的挑战,这很大程度上归因于训练数据集的有限性。为解决这个问题,我们提出了Multi-Passage to Dialogue(MP2D)数据生成框架,这是一种自动创建主题转移对话数据集的数据生成框架。通过利用知识图谱中实体之间的关系,MP2D有效地映射对话中主题的流动,有效地反映了人类对话的动态。它通过 passage-to-dialogue 方法检索相关段落,并将它们转换为对话。通过数量和质量实验,我们证明了MP2D在生成自然主题转移对话方面的有效性。此外,本研究还引入了一个新的主题转移对话基准,TS-WikiDialog。利用数据集,我们证明了即使是大型语言模型(LLMs)也无法有效地处理对话中的主题转移,并展示了基于MP2D训练的主题转移对话任务的性能改进。
https://arxiv.org/abs/2403.05814