Mining predicate-based entailment rules using deep contextual architecture

Semantic inference plays an essential role in numerous Natural Language Processing (NLP) tasks, such as question answering, machine reading and text summarization. Reasoning in natural language is inseparable from the knowledge about inference, which is often represented in the form of predicate-based entailment rules. Many efforts have been dedicated to extracting entailment rules from text corpora by utilizing statistical methodology including distributional hypothesis and Latent Dirichlet Allocation (LDA). However, these studies could not give equal consideration to both coverage and accuracy of the mined rules, which brings instability to downstream applications. To solve this problem, this paper proposes a novel model named Deep Contextual Architecture (DCA), which is driven by Deep Belief Networks (DBNs), for the task of mining predicate-based inference rules from texts. Besides previously used statistical contextual information, we also involve semantic meanings represented by word embeddings into DBNs to learn topic level representation of predicates. Combining benefits from both kinds of information, the proposed DCA model shows potential for better modeling the context of predicates. Evaluation on public datasets demonstrates that our method outperforms several strong baselines.