Path-based reasoning approach for knowledge graph completion using CNN-BiLSTM with attention mechanism

Abstract

Knowledge graphs are valuable resources for building intelligent systems such as question answering or recommendation systems. However, most knowledge graphs are impaired by missing relationships between entities. Embedding methods that translate entities and relations into a low-dimensional space achieve great results, but they only focus on the direct relations between entities and neglect the presence of path relations in graphs. On the contrary, path-based embedding methods consider a single path to make inferences. It also relies on simple recurrent neural networks while highly efficient neural network models are available for processing sequence data. We propose a new approach for knowledge graph completion that combines bidirectional long short-term memory (BiLSTM) and convolutional neural network modules with an attention mechanism. Given a candidate relation and two entities, we encode paths that connect the entities into a low-dimensional space using a convolutional operation followed by BiLSTM. Then, an attention layer is applied to capture the semantic correlation between a candidate relation and each path between two entities and attentively extract reasoning evidence from the representation of multiple paths to predict whether the entities should be connected by the candidate relation. We extend our model to perform multistep reasoning over path representations in an embedding space. A recurrent neural network is designed to repeatedly interact with an attention module to derive logical inference from the representation of multiple paths. We perform link prediction tasks on several knowledge graphs and show that our method achieves better performance compared with recent state-of-the-art path-reasoning methods. (C) 2019 Elsevier Ltd. All rights reserved.