Knowledge_Graph
-
How to Turn Your Knowledge Graph Embeddings into Generative Models via Probabilistic Circuits
2023-05-25 11:30:27Lorenzo Loconte, Nicola Di Mauro, Robert Peharz, Antonio VergariAbstract
Some of the most successful knowledge graph embedding (KGE) models for link prediction -- CP, RESCAL, TuckER, ComplEx -- can be interpreted as energy-based models. Under this perspective they are not amenable for exact maximum-likelihood estimation (MLE), sampling and struggle to integrate logical constraints. This work re-interprets the score functions of these KGEs as circuits -- constrained computational graphs allowing efficient marginalisation. Then, we design two recipes to obtain efficient generative circuit models by either restricting their activations to be non-negative or squaring their outputs. Our interpretation comes with little or no loss of performance for link prediction, while the circuits framework unlocks exact learning by MLE, efficient sampling of new triples, and guarantee that logical constraints are satisfied by design. Furthermore, our models scale more gracefully than the original KGEs on graphs with millions of entities.
Abstract (translated)
这些最成功的知识图嵌入(KGE)模型——CP、RESCAL、TuckER、ComplEx——可以被解释为基于能量模型的模型。从这个角度来看，这些模型并不适合进行精确的最大似然估计(MLE)、采样和集成逻辑约束。这项工作重新解释这些KGE的得分函数，将其解释为电路——具有约束的计算图，可以高效地分配。然后，我们设计了两个食谱，通过限制其激活值是非负或平方其输出，获得高效的生成电路模型。我们的解释对于链接预测没有或几乎没有损失，而电路框架解锁MLE的学习，高效地采样新的三元组，并保证设计所满足的逻辑约束。此外，我们模型在具有数百万实体的Graph上比原始的KGE模型更加平滑地扩展。
URL
https://arxiv.org/abs/2305.15944
PDF
-
BUCA: A Binary Classification Approach to Unsupervised Commonsense Question Answering
2023-05-25 10:59:47Jie He, Simon Chi Lok U, Víctor Gutiérrez-Basulto, Jeff Z. PanAbstract
Unsupervised commonsense reasoning (UCR) is becoming increasingly popular as the construction of commonsense reasoning datasets is expensive, and they are inevitably limited in their scope. A popular approach to UCR is to fine-tune language models with external knowledge (e.g., knowledge graphs), but this usually requires a large number of training examples. In this paper, we propose to transform the downstream multiple choice question answering task into a simpler binary classification task by ranking all candidate answers according to their reasonableness. To this end, for training the model, we convert the knowledge graph triples into reasonable and unreasonable texts. Extensive experimental results show the effectiveness of our approach on various multiple choice question answering benchmarks. Furthermore, compared with existing UCR approaches using KGs, ours is less data hungry. Our code is available at this https URL.
Abstract (translated)
无监督常识推理(UCR)正在变得越来越流行，因为构建常识推理数据集的成本很高，不可避免地会受到限制。UCR的一个流行的方法是通过外部知识(例如知识图谱)优化语言模型，但这通常需要大量训练示例。在本文中，我们提议将后续多项选择回答任务转换为更简单的二进制分类任务，通过按合理性排序所有备选答案来这样做。为了训练模型，我们将知识图谱三元组转换为合理和不合理的文本。广泛的实验结果表明，我们的方法在各种多项选择回答基准测试中的有效性。此外，与使用KGs的现有UCR方法相比，我们的方法的数据需求较少。我们的代码可用在此处https://github.com/lihaoyi21/UCR代码库中。
URL
https://arxiv.org/abs/2305.15932
PDF
-
Collective Knowledge Graph Completion with Mutual Knowledge Distillation
2023-05-25 09:49:40Weihang Zhang, Ovidiu Serban, Jiahao Sun, Yi-ke GuoAbstract
Knowledge graph completion (KGC), the task of predicting missing information based on the existing relational data inside a knowledge graph (KG), has drawn significant attention in recent years. However, the predictive power of KGC methods is often limited by the completeness of the existing knowledge graphs from different sources and languages. In monolingual and multilingual settings, KGs are potentially complementary to each other. In this paper, we study the problem of multi-KG completion, where we focus on maximizing the collective knowledge from different KGs to alleviate the incompleteness of individual KGs. Specifically, we propose a novel method called CKGC-CKD that uses relation-aware graph convolutional network encoder models on both individual KGs and a large fused KG in which seed alignments between KGs are regarded as edges for message propagation. An additional mutual knowledge distillation mechanism is also employed to maximize the knowledge transfer between the models of "global" fused KG and the "local" individual KGs. Experimental results on multilingual datasets have shown that our method outperforms all state-of-the-art models in the KGC task.
Abstract (translated)
知识图 completion (KGC)，即基于知识图(KG)内现有的关系数据预测缺失信息的任务，近年来吸引了大量关注。然而，KGC方法的预测能力往往受到来自不同来源和语言的最新知识图的完整性限制。在单语和多语环境下，KGs可能互相补充。在本文中，我们研究了多KG completion问题，我们重点是最大限度地利用不同KG中的集体知识来减轻个体KG的不完整性。具体来说，我们提出了一种名为CKGC-CKD的新方法，该方法在个体KG和大型合并KG上使用关系 aware 的图卷积神经网络编码模型，并将KG之间的种子对齐视为消息传播的边。此外，我们还采用了额外的互相知识蒸馏机制，以最大限度地促进“全球”合并KG上的“本地”个体KG之间的知识转移。在多语数据集上的实验结果显示，我们的方法在KGC任务中优于所有最先进的模型。
URL
https://arxiv.org/abs/2305.15895
PDF
-
T2TD: Text-3D Generation Model based on Prior Knowledge Guidance
2023-05-25 06:05:52Weizhi Nie, Ruidong Chen, Weijie Wang, Bruno Lepri, Nicu SebeAbstract
In recent years, 3D models have been utilized in many applications, such as auto-driver, 3D reconstruction, VR, and AR. However, the scarcity of 3D model data does not meet its practical demands. Thus, generating high-quality 3D models efficiently from textual descriptions is a promising but challenging way to solve this problem. In this paper, inspired by the ability of human beings to complement visual information details from ambiguous descriptions based on their own experience, we propose a novel text-3D generation model (T2TD), which introduces the related shapes or textual information as the prior knowledge to improve the performance of the 3D generation model. In this process, we first introduce the text-3D knowledge graph to save the relationship between 3D models and textual semantic information, which can provide the related shapes to guide the target 3D model generation. Second, we integrate an effective causal inference model to select useful feature information from these related shapes, which removes the unrelated shape information and only maintains feature information that is strongly relevant to the textual description. Meanwhile, to effectively integrate multi-modal prior knowledge into textual information, we adopt a novel multi-layer transformer structure to progressively fuse related shape and textual information, which can effectively compensate for the lack of structural information in the text and enhance the final performance of the 3D generation model. The final experimental results demonstrate that our approach significantly improves 3D model generation quality and outperforms the SOTA methods on the text2shape datasets.
Abstract (translated)
近年来，三维模型被广泛应用于许多应用，例如自动驾驶、三维重建、虚拟现实和增强现实等。然而，三维模型数据的稀缺性并没有满足其实际需求。因此，从文本描述中生成高质量三维模型是一个有前途但具有挑战性的方法来解决这个问题。在本文中，基于人类从不确定描述中补充视觉信息细节的能力，我们提出了一种新的文本-三维生成模型(T2TD)，该模型引入了相关的形状或文本信息作为先验知识，以提高三维生成模型的性能。在这个过程中，我们首先介绍了文本-三维知识图，以保存三维模型和文本语义信息之间的关系，可以提供相关的形状来指导目标三维模型生成。其次，我们集成了有效的因果推断模型，从这些相关的形状中选择有用的特征信息，删除了不相关的形状信息，仅保留与文本描述密切相关的特征信息。同时，为了有效地将多模态先验知识集成到文本信息中，我们采用了一种新的多层Transformer结构，逐步融合相关的形状和文本信息，可以 effectively弥补文本中的结构信息缺失，并提高三维生成模型的最终性能。最终的实验结果显示，我们的方法显著提高了三维模型生成质量，在文本2shape数据集上优于最先进的方法。
URL
https://arxiv.org/abs/2305.15753
PDF
-
Text-Augmented Open Knowledge Graph Completion via Pre-Trained Language Models
2023-05-24 22:09:35Pengcheng Jiang, Shivam Agarwal, Bowen Jin, Xuan Wang, Jimeng Sun, Jiawei HanAbstract
The mission of open knowledge graph (KG) completion is to draw new findings from known facts. Existing works that augment KG completion require either (1) factual triples to enlarge the graph reasoning space or (2) manually designed prompts to extract knowledge from a pre-trained language model (PLM), exhibiting limited performance and requiring expensive efforts from experts. To this end, we propose TAGREAL that automatically generates quality query prompts and retrieves support information from large text corpora to probe knowledge from PLM for KG completion. The results show that TAGREAL achieves state-of-the-art performance on two benchmark datasets. We find that TAGREAL has superb performance even with limited training data, outperforming existing embedding-based, graph-based, and PLM-based methods.
Abstract (translated)
开放知识图谱(KG)的完成的使命是从已知的事实中得出新发现。现有用于增强KG完成的工作时，要么需要扩展图推理空间使用事实三元组，要么需要手动设计 prompts 以从训练好的语言模型(PLM)中提取知识，表现出性能有限，并需要专家昂贵的努力。为此，我们提出了TAGReal，它自动生成高质量的查询提示并从大型文本库中提取支持信息，以从PLM中获取知识，用于KG completion。结果表明，TAGReal在两个基准数据集上实现了最先进的性能。我们发现，即使训练数据有限，TAGReal仍然表现出优异的性能，优于现有的嵌入型、基于图和PLM的方法。
URL
https://arxiv.org/abs/2305.15597
PDF
-
The Role of Output Vocabulary in T2T LMs for SPARQL Semantic Parsing
2023-05-24 12:55:04Debayan Banerjee, Pranav Ajit Nair, Ricardo Usbeck, Chris BiemannAbstract
In this work, we analyse the role of output vocabulary for text-to-text (T2T) models on the task of SPARQL semantic parsing. We perform experiments within the the context of knowledge graph question answering (KGQA), where the task is to convert questions in natural language to the SPARQL query language. We observe that the query vocabulary is distinct from human vocabulary. Language Models (LMs) are pre-dominantly trained for human language tasks, and hence, if the query vocabulary is replaced with a vocabulary more attuned to the LM tokenizer, the performance of models may improve. We carry out carefully selected vocabulary substitutions on the queries and find absolute gains in the range of 17% on the GrailQA dataset.
Abstract (translated)
在本研究中，我们对文本到文本(T2T)模型中的输出词汇表在SPARQL语义解析任务中的作用进行了分析。我们在知识图问答(KGQA)的背景下进行了实验，该任务是将自然语言问题转换为SPARQL查询语言。我们观察到，查询词汇与人类词汇存在显著差异。语言模型(LMs)主要是针对人类语言任务进行训练的，因此，如果查询词汇被替换为更适应LM分割的单词，模型性能可能会提高。我们对查询进行了精心筛选的词汇替换，并在GrailQA数据集上发现了17%的绝对增益。
URL
https://arxiv.org/abs/2305.15108
PDF
-
A RelEntLess Benchmark for Modelling Graded Relations between Named Entities
2023-05-24 10:41:24Asahi Ushio, Jose Camacho Collados, Steven SchockaertAbstract
Relations such as "is influenced by", "is known for" or "is a competitor of" are inherently graded: we can rank entity pairs based on how well they satisfy these relations, but it is hard to draw a line between those pairs that satisfy them and those that do not. Such graded relations play a central role in many applications, yet they are typically not covered by existing Knowledge Graphs. In this paper, we consider the possibility of using Large Language Models (LLMs) to fill this gap. To this end, we introduce a new benchmark, in which entity pairs have to be ranked according to how much they satisfy a given graded relation. The task is formulated as a few-shot ranking problem, where models only have access to a description of the relation and five prototypical instances. We use the proposed benchmark to evaluate state-of-the-art relation embedding strategies as well as several recent LLMs, covering both publicly available LLMs and closed models such as GPT-4. Overall, we find a strong correlation between model size and performance, with smaller Language Models struggling to outperform a naive baseline. The results of the largest Flan-T5 and OPT models are remarkably strong, although a clear gap with human performance remains.
Abstract (translated)
关系如“受到影响”、“著称于”或“是竞争对手”等是 inherently graded 的：我们可以根据它们是否满足这些关系来对实体 pairs 进行排序，但很难在满足这些关系的实体 pairs 和不满足这些关系的实体 pairs 之间划一条线。在许多应用中，这种 graded 关系扮演着关键角色，但它们通常未被现有知识图覆盖。在本文中，我们考虑使用大型语言模型(LLM)来填补这一空缺。为此，我们引入了一个新的基准，在该基准中，实体 pairs 必须根据它们是否满足给定的grading relation 进行排序。任务被设计为一个简单的问题，模型只能访问关系的描述和五个典型的实例。我们使用 proposed 基准来评估最先进的关系嵌入策略以及最近发布的几个LLM，包括公开可用的LLM和如GPT-4这样的闭式模型。总体而言，我们发现模型大小和性能之间存在强烈的相关性，较小的语言模型一直在努力超越简单的基准。最大的 Flan-T5 和 OPT 模型的结果非常显著，尽管与人类表现仍然存在明显的差距。
URL
https://arxiv.org/abs/2305.15002
PDF
-
Revisit and Outstrip Entity Alignment: A Perspective of Generative Models
2023-05-24 02:39:20Lingbing Guo, Zhuo Chen, Jiaoyan Chen, Huajun ChenAbstract
Recent embedding-based methods have achieved great successes on exploiting entity alignment from knowledge graph (KG) embeddings of multiple modals. In this paper, we study embedding-based entity alignment (EEA) from a perspective of generative models. We show that EEA is a special problem where the main objective is analogous to that in a typical generative model, based on which we theoretically prove the effectiveness of the recently developed generative adversarial network (GAN)-based EEA methods. We then reveal that their incomplete objective limits the capacity on both entity alignment and entity synthesis (i.e., generating new entities). We mitigate this problem by introducing a generative EEA (abbr., GEEA) framework with the proposed mutual variational autoencoder (M-VAE) as the generative model. M-VAE can convert an entity from one KG to another and generate new entities from random noise vectors. We demonstrate the power of GEEA with theoretical analysis and empirical experiments on both entity alignment and entity synthesis tasks.
Abstract (translated)
最近基于嵌入的方法在利用多模式知识图嵌入的实体对齐方面取得了巨大的成功。在本文中，我们从一个生成模型的角度来看研究基于嵌入的实体对齐(EEA)问题。我们表明，EEA是一个特殊的问题，其主要目标是类似于一个典型的生成模型的目标，基于这个目标，我们理论上证明了最近开发的生成对抗网络(GAN)based EEA方法的有效性。然后我们揭示了他们的不完整目标限制了实体对齐和实体生成(即生成新实体)的能力。我们通过引入一个生成EEA(暂称为GEEA)框架，以 proposed 的共变自编码器(M-VAE)作为生成模型，将从一个KG转换到另一个KG，并从随机噪声向量生成新实体。我们从理论上分析和实验上证明了GEEA的力量，同时解决了实体对齐和实体生成任务中的实体对齐和生成任务。
URL
https://arxiv.org/abs/2305.14651
PDF
-
Learning to Generate Novel Scientific Directions with Contextualized Literature-based Discovery
2023-05-23 17:12:08Qingyun Wang, Doug Downey, Heng Ji, Tom HopeAbstract
Literature-Based Discovery (LBD) aims to discover new scientific knowledge by mining papers and generating hypotheses. Standard LBD is limited to predicting pairwise relations between discrete concepts (e.g., drug-disease links). LBD also ignores critical contexts like experimental settings (e.g., a specific patient population where a drug is evaluated) and background knowledge and motivations that human scientists consider (e.g., to find a drug candidate without specific side effects). We address these limitations with a novel formulation of contextualized-LBD (C-LBD): generating scientific hypotheses in natural language, while grounding them in a context that controls the hypothesis search space. We present a new modeling framework using retrieval of ``inspirations'' from a heterogeneous network of citations and knowledge graph relations, and create a new dataset derived from papers. In automated and human evaluations, our models improve over baselines, including powerful large language models (LLMs), but also reveal challenges on the road to building machines that generate new scientific knowledge.
Abstract (translated)
文献发现(LBD)旨在通过挖掘论文并生成假设来发现新的科学知识。标准LBD只能预测离散概念之间的一对一关系(例如，药物-疾病联系)。LBD也忽略了实验设置(例如，一个特定患者群体评估药物)、人类科学家考虑的背景知识和动机(例如，找到没有特定副作用的药物候选者)等关键上下文。我们采用了一种新的上下文化LBD(C-LBD) formulation，通过从引用和知识图关联网络中的异质性网络中检索“灵感”，提出了一种新的建模框架，并使用该框架从论文中创建了一个新的数据集。在自动化和人类评估中，我们的模型比基准模型更美好，包括强大的大型语言模型(LLMs)，但也揭示了在构建生成新知识的机器方面所面临的挑战。
URL
https://arxiv.org/abs/2305.14259
PDF
-
Complementing GPT-3 with Few-Shot Sequence-to-Sequence Semantic Parsing over Wikidata
2023-05-23 16:20:43Silei Xu, Theo Culhane, Meng-Hsi Wu, Sina J. Semnani, Monica S. LamAbstract
As the largest knowledge base, Wikidata is a massive source of knowledge, complementing large language models with well-structured data. In this paper, we present WikiWebQuestions, a high-quality knowledge base question answering benchmark for Wikidata. This new benchmark uses real-world human data with SPARQL annotation to facilitate a more accurate comparison with large language models utilizing the up-to-date answers from Wikidata. Additionally, a baseline for this benchmark is established with an effective training data synthesis methodology and WikiSP, a Seq2Seq semantic parser, that handles large noisy knowledge graphs. Experimental results illustrate the effectiveness of this methodology, achieving 69% and 59% answer accuracy in the dev set and test set, respectively. We showed that we can pair semantic parsers with GPT-3 to provide a combination of verifiable results and qualified guesses that can provide useful answers to 97% of the questions in the dev set of our benchmark.
Abstract (translated)
作为一家最大的知识库，维基百科是一个巨大的知识来源，与大型语言模型结合，提供了良好的数据结构。在本文中，我们介绍了维基百科Web问题，这是一个高质量的知识库问题回答基准维基百科。这个新基准使用现实世界的人形数据，并使用SPARQL注释，以促进更准确地比较使用维基百科最新答案的大型语言模型。此外，该基准的基础线通过有效的训练数据合成方法和维基百科SP，一个Seq2Seq语义解析器，处理大型噪声知识图。实验结果显示该方法的有效性，在开发集和测试集上分别实现69%和59%的答案准确性。我们表明，我们可以将语义解析器和GPT-3配对，提供可验证的结果和 qualified guesses，为基准开发集上的97%问题提供有用的答案。
URL
https://arxiv.org/abs/2305.14202
PDF
-
To Copy Rather Than Memorize: A Vertical Learning Paradigm for Knowledge Graph Completion
2023-05-23 14:53:20Rui Li, Xu Chen, Chaozhuo Li, Yanming Shen, Jianan Zhao, Yujing Wang, Weihao Han, Hao Sun, Weiwei Deng, Qi Zhang, Xing XieAbstract
Embedding models have shown great power in knowledge graph completion (KGC) task. By learning structural constraints for each training triple, these methods implicitly memorize intrinsic relation rules to infer missing links. However, this paper points out that the multi-hop relation rules are hard to be reliably memorized due to the inherent deficiencies of such implicit memorization strategy, making embedding models underperform in predicting links between distant entity pairs. To alleviate this problem, we present Vertical Learning Paradigm (VLP), which extends embedding models by allowing to explicitly copy target information from related factual triples for more accurate prediction. Rather than solely relying on the implicit memory, VLP directly provides additional cues to improve the generalization ability of embedding models, especially making the distant link prediction significantly easier. Moreover, we also propose a novel relative distance based negative sampling technique (ReD) for more effective optimization. Experiments demonstrate the validity and generality of our proposals on two standard benchmarks. Our code is available at this https URL.
Abstract (translated)
嵌入模型在知识图的完成任务(KGC)中展现出了巨大的能力。通过学习每个训练三元组的结构约束，这些方法会潜意识地记住内在的关系规则来推断缺失的链接。然而，本文指出，多级关系规则很难可靠地记忆，因为这种类型的隐含记忆策略固有的缺陷，导致嵌入模型在预测距离实体 pairs 的链接时表现不佳。为了解决这一问题，我们提出了垂直学习范式(VLP)，它扩展了嵌入模型，允许从相关事实三元组中 explicitly 复制目标信息，以更准确地预测链接。而不仅仅是依赖隐含记忆，VLP 直接提供了额外的提示，以提高嵌入模型的泛化能力，特别是使远程链接预测变得更容易。此外，我们还提出了一种新颖的相对距离基于负采样技术(ReD)，以更有效的优化。实验证明了我们提出的建议的两个标准基准的 validity 和 generalizability。我们的代码现在可以在这个 https URL 上可用。
URL
https://arxiv.org/abs/2305.14126
PDF
-
Message Intercommunication for Inductive Relation Reasoning
2023-05-23 13:51:46Ke Liang, Lingyuan Meng, Sihang Zhou, Siwei Wang, Wenxuan Tu, Yue Liu, Meng Liu, Xinwang LiuAbstract
Inductive relation reasoning for knowledge graphs, aiming to infer missing links between brand-new entities, has drawn increasing attention. The models developed based on Graph Inductive Learning, called GraIL-based models, have shown promising potential for this task. However, the uni-directional message-passing mechanism hinders such models from exploiting hidden mutual relations between entities in directed graphs. Besides, the enclosing subgraph extraction in most GraIL-based models restricts the model from extracting enough discriminative information for reasoning. Consequently, the expressive ability of these models is limited. To address the problems, we propose a novel GraIL-based inductive relation reasoning model, termed MINES, by introducing a Message Intercommunication mechanism on the Neighbor-Enhanced Subgraph. Concretely, the message intercommunication mechanism is designed to capture the omitted hidden mutual information. It introduces bi-directed information interactions between connected entities by inserting an undirected/bi-directed GCN layer between uni-directed RGCN layers. Moreover, inspired by the success of involving more neighbors in other graph-based tasks, we extend the neighborhood area beyond the enclosing subgraph to enhance the information collection for inductive relation reasoning. Extensive experiments on twelve inductive benchmark datasets demonstrate that our MINES outperforms existing state-of-the-art models, and show the effectiveness of our intercommunication mechanism and reasoning on the neighbor-enhanced subgraph.
Abstract (translated)
知识图谱中的基于图归纳学习的关联推理任务，旨在推断新实体之间的缺失链接，已经引起了越来越多的关注。基于图归纳学习的开发模型，称为GraIL-based模型，在这些任务中表现出了很好的潜力。然而，单向的消息传递机制阻碍了这些模型利用定向图中实体之间的隐藏互信息。此外，大多数GraIL-based模型中的外层关系提取限制模型从提取足够的用于推理的特征信息。因此，这些模型的表达能力是有限的。为了解决这些问题，我们提出了一种新的GraIL-based关联推理模型，称为MINES，通过在邻居增强子图引入消息交互机制来实现。具体来说，消息交互机制旨在捕捉未包含的隐藏互信息。它通过在 uni-directed RGCN 层之间插入一个无向/双向GCN层，将连接实体的信息交互引入双向信息交互。此外，受其他基于图的任务中增加邻居成功的经验启发，我们扩展了邻居增强子图周围的邻域，以提高基于归纳的关系推理的信息收集。对十二个基于归纳的基准数据集进行了广泛的实验，结果表明，我们的 MINES 比现有的最先进的模型表现更好，并展示了我们的消息交互机制和推理在邻居增强子图方面的有效性。
URL
https://arxiv.org/abs/2305.14074
PDF
-
Conversational Recommendation as Retrieval: A Simple, Strong Baseline
2023-05-23 06:21:31Raghav Gupta, Renat Aksitov, Samrat Phatale, Simral Chaudhary, Harrison Lee, Abhinav RastogiAbstract
Conversational recommendation systems (CRS) aim to recommend suitable items to users through natural language conversation. However, most CRS approaches do not effectively utilize the signal provided by these conversations. They rely heavily on explicit external knowledge e.g., knowledge graphs to augment the models' understanding of the items and attributes, which is quite hard to scale. To alleviate this, we propose an alternative information retrieval (IR)-styled approach to the CRS item recommendation task, where we represent conversations as queries and items as documents to be retrieved. We expand the document representation used for retrieval with conversations from the training set. With a simple BM25-based retriever, we show that our task formulation compares favorably with much more complex baselines using complex external knowledge on a popular CRS benchmark. We demonstrate further improvements using user-centric modeling and data augmentation to counter the cold start problem for CRSs.
Abstract (translated)
对话推荐系统(CRS)旨在通过自然语言对话向用户推荐适合的物品。然而，大多数CRS方法并未有效地利用这些对话提供的信号。它们 heavily 依赖显式外部知识，例如知识图谱，以增加模型对物品和属性的理解，这很难实现规模扩展。为了减轻这种情况，我们提出了一种与信息检索(IR)方法相关的替代方法，该方法将对话表示为查询和物品作为要检索的文档。我们扩展了从训练集使用对话进行检索的文档表示。使用简单的BM25检索器，我们展示了我们的任务 formulation 与在一个非常流行的CRS基准上使用更复杂的基线通过复杂的外部知识进行的比较。我们使用用户中心建模和数据增强来对抗CRS系统的 cold start problem。
URL
https://arxiv.org/abs/2305.13725
PDF
-
Query Structure Modeling for Inductive Logical Reasoning Over Knowledge Graphs
2023-05-23 01:25:29Siyuan Wang, Zhongyu Wei, Meng Han, Zhihao Fan, Haijun Shan, Qi Zhang, Xuanjing HuangAbstract
Logical reasoning over incomplete knowledge graphs to answer complex logical queries is a challenging task. With the emergence of new entities and relations in constantly evolving KGs, inductive logical reasoning over KGs has become a crucial problem. However, previous PLMs-based methods struggle to model the logical structures of complex queries, which limits their ability to generalize within the same structure. In this paper, we propose a structure-modeled textual encoding framework for inductive logical reasoning over KGs. It encodes linearized query structures and entities using pre-trained language models to find answers. For structure modeling of complex queries, we design stepwise instructions that implicitly prompt PLMs on the execution order of geometric operations in each query. We further separately model different geometric operations (i.e., projection, intersection, and union) on the representation space using a pre-trained encoder with additional attention and maxout layers to enhance structured modeling. We conduct experiments on two inductive logical reasoning datasets and three transductive datasets. The results demonstrate the effectiveness of our method on logical reasoning over KGs in both inductive and transductive settings.
Abstract (translated)
在不完整的知识图谱上进行逻辑推理以回答复杂的逻辑查询是一项挑战性的任务。随着不断进化的KGs中新实体和新关系的出现，对KG上的基于归纳的逻辑推理 became a crucial问题。然而，以前的基于PLM的方法 struggle 来建模复杂的查询逻辑结构，这限制了他们在同结构内泛化的能力。在本文中，我们提出了一种基于结构建模的文字编码框架，以处理KG上的基于归纳的逻辑推理。它使用预先训练的语言模型来线性化查询结构和实体，以找到答案。为了建模复杂的查询结构，我们设计了一系列步骤指令，暗示PLM在每个查询中的几何操作执行顺序。我们还使用一个带有额外注意力和最大输出层的预先训练编码器来分别建模不同的几何操作，以增强结构建模。我们研究了两个基于归纳的逻辑推理数据集和三个基于转换的数据集。结果证明了我们在基于和基于转换 settings 下对KG上的逻辑推理推理的有效性。
URL
https://arxiv.org/abs/2305.13585
PDF
-
NeSy4VRD: A Multifaceted Resource for Neurosymbolic AI Research using Knowledge Graphs in Visual Relationship Detection
2023-05-22 17:28:25David Herron, Ernesto Jiménez-Ruiz, Giacomo Tarroni, Tillman WeydeAbstract
NeSy4VRD is a multifaceted resource designed to support the development of neurosymbolic AI (NeSy) research. NeSy4VRD re-establishes public access to the images of the VRD dataset and couples them with an extensively revised, quality-improved version of the VRD visual relationship annotations. Crucially, NeSy4VRD provides a well-aligned, companion OWL ontology that describes the dataset this http URL comes with open source infrastructure that provides comprehensive support for extensibility of the annotations (which, in turn, facilitates extensibility of the ontology), and open source code for loading the annotations to/from a knowledge graph. We are contributing NeSy4VRD to the computer vision, NeSy and Semantic Web communities to help foster more NeSy research using OWL-based knowledge graphs.
Abstract (translated)
NeSy4VRD是一个多用途资源，旨在支持神经符号人工智能(NeSy)的研究。它重新恢复了VRD数据集的图像公开访问，并将它们与经过广泛修订和提高质量的VRD视觉关系注释版本相结合。关键地，NeSy4VRD提供了一组配合默契的OWL本体论，该本体论描述了这个http URL所包含的该数据集，提供了全面支持扩展注释(这反过来促进了本体论的扩展)的开源基础设施，以及用于将注释加载到知识图谱上的开源代码。我们正在向计算机视觉、NeSy和语义网社区贡献NeSy4VRD，以帮助促进基于OWL的知识图谱的更多NeSy研究。
URL
https://arxiv.org/abs/2305.13258
PDF
-
LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities
2023-05-22 15:56:44Yuqi Zhu, Xiaohan Wang, Jing Chen, Shuofei Qiao, Yixin Ou, Yunzhi Yao, Shumin Deng, Huajun Chen, Ningyu ZhangAbstract
This paper presents an exhaustive quantitative and qualitative evaluation of Large Language Models (LLMs) for Knowledge Graph (KG) construction and reasoning. We employ eight distinct datasets that encompass aspects including entity, relation and event extraction, link prediction, and question answering. Empirically, our findings suggest that GPT-4 outperforms ChatGPT in the majority of tasks and even surpasses fine-tuned models in certain reasoning and question-answering datasets. Moreover, our investigation extends to the potential generalization ability of LLMs for information extraction, which culminates in the presentation of the Virtual Knowledge Extraction task and the development of the VINE dataset. Drawing on these empirical findings, we further propose AutoKG, a multi-agent-based approach employing LLMs for KG construction and reasoning, which aims to chart the future of this field and offer exciting opportunities for advancement. We anticipate that our research can provide invaluable insights for future undertakings of KG\footnote{Code and datasets will be available in this https URL.
Abstract (translated)
本论文全面评估了大型语言模型(LLM)用于知识图(KG)建设和推理的量化和定性评价。我们使用了8个不同的数据集，涵盖了实体、关系和事件提取、链接预测和问答等方面。经验表明，我们的研究结果表明GPT-4在大多数任务中比ChatGPT表现更好，甚至在一些推理和问答数据集上超过优化模型。此外，我们的研究还扩展到LLM的信息提取潜在 generalization能力，这最终导致了虚拟知识提取任务和VINE数据集的发布。基于这些经验成果，我们进一步提出了AutoKG，一种基于多代理的方法来使用LLM进行KG建设和推理，旨在预测该领域的未来并提供令人兴奋的进步机会。我们预计，我们的研究可以为KG领域的未来任务提供宝贵的见解。
URL
https://arxiv.org/abs/2305.13168
PDF
-
Friendly Neighbors: Contextualized Sequence-to-Sequence Link Prediction
2023-05-22 14:16:45Adrian Kochsiek, Apoorv Saxena, Inderjeet Nair, Rainer GemullaAbstract
We propose KGT5-context, a simple sequence-to-sequence model for link prediction (LP) in knowledge graphs (KG). Our work expands on KGT5, a recent LP model that exploits textual features of the KG, has small model size, and is scalable. To reach good predictive performance, however, KGT5 relies on an ensemble with a knowledge graph embedding model, which itself is excessively large and costly to use. In this short paper, we show empirically that adding contextual information - i.e., information about the direct neighborhood of a query vertex - alleviates the need for a separate KGE model to obtain good performance. The resulting KGT5-context model obtains state-of-the-art performance in our experimental study, while at the same time reducing model size significantly.
Abstract (translated)
我们提出KGT5-context，这是一种简单的序列到序列模型，用于知识图谱(KG)中的链接预测(LP)。我们的研究扩展了KGT5，这是一种最近使用的LP模型，利用KG中的文本特征，具有小型模型规模，并且可以扩展。然而，要获得良好的预测性能，KGT5依赖于一个知识图谱嵌入模型的集成，其本身过大且使用成本较高。在这篇论文中，我们经验证地表明，添加上下文信息(即查询顶点的直接邻居信息)可以消除使用单独的KGE模型获得良好性能的需求。因此，我们得到的KGT5-context模型在我们的实验研究中获得了最先进的性能，同时显著减少了模型规模。
URL
https://arxiv.org/abs/2305.13059
PDF
-
3D Rotation and Translation for Hyperbolic Knowledge Graph Embedding
2023-05-22 13:17:23Yihua Zhu, Hidetoshi ShimodairaAbstract
The main objective of Knowledge Graph (KG) embeddings is to learn low-dimensional representations of entities and relations, enabling the prediction of missing facts. A significant challenge in achieving better KG embeddings lies in capturing relation patterns, including symmetry, antisymmetry, inversion, commutative composition, non-commutative composition, hierarchy, and multiplicity. This study introduces a novel model called 3H-TH (3D Rotation and Translation in Hyperbolic space) that captures these relation patterns simultaneously. In contrast, previous attempts have not achieved satisfactory performance across all the mentioned properties at the same time. The experimental results demonstrate that the new model outperforms existing state-of-the-art models in terms of accuracy, hierarchy property, and other relation patterns in low-dimensional space, meanwhile performing similarly in high-dimensional space.
Abstract (translated)
知识图(KG)嵌入的主要目标是学习实体和关系的低维表示，实现对缺失事实的预测。实现更好的KG嵌入的一个 significant 挑战是捕获关系模式，包括对称性、反对称性、翻转、互操作性组合、非互操作性组合、层次和多重性。本研究介绍了一种名为3H-TH(3D旋转和Translation in Hyperbolic Space)的新模型，它可以同时捕捉这些关系模式。与前几次尝试不同，之前的方法没有同时实现所有提到的属性的满意表现。实验结果表明，新模型在低维空间中的准确性、层次属性和其他关系模式方面表现更好，而在高维空间中类似地表现。
URL
https://arxiv.org/abs/2305.13015
PDF
-
Evaluating Prompt-based Question Answering for Object Prediction in the Open Research Knowledge Graph
2023-05-22 10:35:18Jennifer D'Souza, Moussab Hrou, Sören AuerAbstract
There have been many recent investigations into prompt-based training of transformer language models for new text genres in low-resource settings. The prompt-based training approach has been found to be effective in generalizing pre-trained or fine-tuned models for transfer to resource-scarce settings. This work, for the first time, reports results on adopting prompt-based training of transformers for \textit{scholarly knowledge graph object prediction}. The work is unique in the following two main aspects. 1) It deviates from the other works proposing entity and relation extraction pipelines for predicting objects of a scholarly knowledge graph. 2) While other works have tested the method on text genera relatively close to the general knowledge domain, we test the method for a significantly different domain, i.e. scholarly knowledge, in turn testing the linguistic, probabilistic, and factual generalizability of these large-scale transformer models. We find that (i) per expectations, transformer models when tested out-of-the-box underperform on a new domain of data, (ii) prompt-based training of the models achieve performance boosts of up to 40\% in a relaxed evaluation setting, and (iii) testing the models on a starkly different domain even with a clever training objective in a low resource setting makes evident the domain knowledge capture gap offering an empirically-verified incentive for investing more attention and resources to the scholarly domain in the context of transformer models.
Abstract (translated)
有许多最近的研究涉及到在资源匮乏的环境下对新的文本类型进行Transformer语言模型的即时训练。即时训练方法被发现可以有效地将预先训练或优化的模型应用于需要大量资源的情境。这项工作是独特的，主要有两个方面。1)它与其他工作提出了实体和关系提取管道，用于预测学术知识图对象。2)虽然其他工作在文本类相对较为通用的知识领域进行了测试，我们测试了一个非常不同的领域，即学术知识领域，并测试这些大型Transformer模型的语言学、概率和事实 generalizability。我们发现(i)按照期望，在测试之外的性能表现不佳，在一个新数据域中，(ii)即时训练模型在放松评估设置下实现性能提升，高达40\%,(iii)即使在资源匮乏的环境下，即使有聪明的训练目标，在一个非常不同的领域进行测试，仍表明了领域知识捕获的差距，提供了经验证的激励，在Transformer模型中更多地关注学术领域的资源投入。
URL
https://arxiv.org/abs/2305.12900
PDF
-
GraphCare: Enhancing Healthcare Predictions with Open-World Personalized Knowledge Graphs
2023-05-22 07:35:43Pengcheng Jiang, Cao Xiao, Adam Cross, Jimeng SunAbstract
Clinical predictive models often rely on patients electronic health records (EHR), but integrating medical knowledge to enhance predictions and decision-making is challenging. This is because personalized predictions require personalized knowledge graphs (KGs), which are difficult to generate from patient EHR data. To address this, we propose GraphCare, an open-world framework that leverages external KGs to improve EHR-based predictions. Our method extracts knowledge from large language models (LLMs) and external biomedical KGs to generate patient-specific KGs, which are then used to train our proposed Bi-attention AugmenTed BAT graph neural network GNN for healthcare predictions. We evaluate GraphCare on two public datasets: MIMIC-III and MIMIC-IV. Our method outperforms baseline models in four vital healthcare prediction tasks: mortality, readmission, length-of-stay, and drug recommendation, improving AUROC on MIMIC-III by average margins of 10.4%, 3.8%, 2.0%, and 1.5%, respectively. Notably, GraphCare demonstrates a substantial edge in scenarios with limited data availability. Our findings highlight the potential of using external KGs in healthcare prediction tasks and demonstrate the promise of GraphCare in generating personalized KGs for promoting personalized medicine.
Abstract (translated)
临床预测模型通常依赖于患者的电子健康记录(EHR)，但集成医学知识以增强预测和决策-making是挑战性的。这是因为个性化预测需要个性化知识图(KGs)，从患者EHR数据中难以生成。为了解决这一问题，我们提出了GraphCare，一个开放世界框架，利用外部KGs以提高基于EHR的预测。我们的方法从大型语言模型(LLMs)和外部生物医学KGs中提取知识，生成患者-specific KGs，然后用于训练我们提出的Bi-attention AugmenTed BAT graph neural network GNN，用于 healthcare预测。我们评估了GraphCare，在两个公共数据集上(米氏细胞学(MIMIC-III)和米氏细胞学(MIMIC-IV))进行了四个关键医疗保健预测任务：死亡率、再读、住院时间、药物推荐，平均差距为10.4%、3.8%、2.0%和1.5%。值得注意的是，GraphCare在数据可用性有限的情况下表现出巨大的优势。我们的发现突出了在医疗保健预测任务中使用外部KG的潜力，并展示了GraphCare生成个性化KGs以促进个性化医学的 promise。
URL
https://arxiv.org/abs/2305.12788
PDF
