Abstract
Knowledge graph embedding (KGE) that maps entities and relations into vector representations is essential for downstream tasks. Conventional KGE methods require relatively high-dimensional entity representations to preserve the structural information of knowledge graph, but lead to oversized model parameters. Recent methods reduce model parameters by adopting low-dimensional entity representations, while developing techniques (e.g., knowledge distillation) to compensate for the reduced dimension. However, such operations produce degraded model accuracy and limited reduction of model parameters. Specifically, we view the concatenation of all entity representations as an embedding layer, and then conventional KGE methods that adopt high-dimensional entity representations equal to enlarging the width of the embedding layer to gain expressiveness. To achieve parameter efficiency without sacrificing accuracy, we instead increase the depth and propose a deeper embedding network for entity representations, i.e., a narrow embedding layer and a multi-layer dimension lifting network (LiftNet). Experiments on three public datasets show that the proposed method (implemented based on TransE and DistMult) with 4-dimensional entity representations achieves more accurate link prediction results than counterpart parameter-efficient KGE methods and strong KGE baselines, including TransE and DistMult with 512-dimensional entity representations.
Abstract (translated)
将实体和关系映射为向量表示的知识图嵌入(KGE)对于后续任务是至关重要的。传统的KGE方法需要较高的向量实体表示来保持知识图的结构信息，但会导致模型参数过拟合。最近的方法通过采用低向量实体表示来减少模型参数，同时开发技术(例如知识蒸馏)来补偿减少维度。然而，这种操作会损害模型精度，并限制模型参数的减少。具体来说，我们将所有实体表示拼接起来视为嵌入层，而采用高向量实体表示的传统KGE方法等同于扩大嵌入层的宽度以获得表达能力。为了实现参数效率，而不是牺牲精度，我们会增加深度，并提出更深入的实体嵌入网络，即狭窄的嵌入层和一个多层维度提升网络( liftNet)。在三个公共数据集上的实验表明，采用4向量实体表示的新方法(基于TransE和DistMult)比参数高效的KGE方法和强KGE基线，包括TransE和DistMult使用512向量实体表示的方法，实现更准确的链接预测结果。
URL
https://arxiv.org/abs/2303.12816
