Doctoral Dissertation Research: Socially guided allocation of attention and the memory encoding of spoken language

Part of understanding spoken language involves storing memories of speech. These memories are a central part of a system that allows humans to understand language as quickly and adeptly as they do. For example, when a listener hears a sentence, they can understand something about the meaning and the talker depending on how the sentence is spoken (e.g., mood, emotion, etc.). Memories from past experiences lead individuals to infer these meanings. But humans cannot store every memory with the same detail. Some experiences are remembered clearly while others are forgotten, while others are merged or only stored in part. One open question is how these differences in memory emerge. Recent research has shown that memory encoding is talker- and group-dependent and strongly associated with social traits cued through spoken language. In this project, the researchers investigate the psychological underpinnings of differences in memories for spoken language. Understanding how cognitive processes vary across diverse talkers is important for theory and for society. For example, in the legal system, this work relates to the reliability of witnesses’ memory of speech. In medicine, accurate diagnoses rely on conversations between the doctor and the patient. And, in developing new AI technologies, researchers need to understand how these processes work in humans in order to avoid recreating biases in machines. This doctoral dissertation project tests the hypothesis that listeners attend differently to speech in different contexts, and this results in differences in memory. When the listener attends more, the memory is recorded more strongly. This approach diverges from a broad expectation that the strength of memories for speech is mostly determined by how often listeners hear a word or pronunciation. The researchers propose a model in which listeners’ level of attention is central and is mediated by the social traits of the speaker and the context of the speech. Three main hypotheses are tested. First, speech recognition and memory encoding depend on associations and beliefs listeners have about speakers based on their voices. Second, this process is driven by the listener’s attention, not just how often a pronunciation has been heard. Third, the traits of speech that attract attention are not always the same. Instead, they depend on the context. For each hypothesis, the researchers conduct two corresponding experiments: One experiment focusing on memory for the sounds of speech, and the other focusing on memory for the meanings of sentences. The results of the experiments shed light on how humans understand and remember speech and how these processes interact with a social world. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.