EAGER: Grounding Natural Language Inference in Cognitive Processes

Being able to detect textual similarity is important for many applications including machine translation, detection of plagiarism, text generation, fact checking etc. At the word level, two words tend to mean the same thing when one can be swapped with the other with little or no consequence. But how does this approach extend to the sentence level and beyond? According to an inference-centered view, a significant part of a sentence’s meaning can be understood in terms of the “inferential halo” of each sentence. The “inferential halo” is all the inferences, i.e., implied meanings, that a sentence has. Comparing the semantic similarity of two sentences or text would then be accomplished by comparing all the inferences that each sentence or text implies. However, current Natural Language Processing approaches to detecting sentence and text similarities are limited to measures based on word or substring similarities which do not capture adequately the meaning of a text. This project addresses the limitations of previous approaches a) enriching the notion and representation of inference and b) learning how people naturally reason about semantic relations of texts. The novelty of our approach draws on established work in Cognitive Psychology. For the enrichment of the notion of inference, we introduce the distinction between a) quick and automatic reasoning, known as Type 1, and b) slower and more deliberate reasoning, known as Type 2. Type 1 reasoning applies when for example we recognize a face and Type 2 when we calculate the tip for a bill. To learn how people naturally reason, we collect data using data collection protocols established in Cognitive Psychology for quick and slow reasoning. For data collection, we experiment with a novel level of granularity to better capture the range of inferences made by humans and train new computational models of detecting semantic inferences. The proposed research will yield results, guidelines, and new computational models that will lead to a) a novel way of studying informal reasoning in language processing and b) improved metrics of textual similarity. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.