REU Site: Computational Methods for Understanding Music, Media and Minds

How can a computer be trained to discover musical motifs? How does the brain encode musical features when people listen to music and associate it with their memories? Can we leverage recent advances in artificial intelligence (AI) to make communication easier between deaf children and their hearing parents? These are some of the questions that students will investigate in this Research Experience for Undergraduates (REU) site at the University of Rochester. Students will explore an exciting, interdisciplinary research area that combines computer science, electrical engineering, cognitive science, and music. Each student will be mentored by two or more faculty members from the University's schools of engineering, arts and sciences, medicine, and music. Other activities of the REU site include workshops on career development; scholarship community colloquiums; graduate school preparation; programming for machine learning; and music-focused activities. The goals of this REU are to broaden the horizons of students engaged in computer science research. The site welcomes students from institutions where opportunities for interdisciplinary research combining computational methods, arts and humanities may be limited. Students who are already majoring in computer science will discover that the research in the field is not limited to traditional applications, but can address questions of art, culture, and human psychology. Students with experience in combining computer science with humanistic research are already in great demand in industry and academia and will help define what it means to be a computer scientist in the 21st century. Students in the University's REU will engage in interdisciplinary research that combines machine learning, computer audition, music theory, and cognitive science. These disciplines are united by their use of a common set of formal representations and computational methods; in particular, probabilistic models and machine learning. Research projects include finding neural encoding mechanisms of music listening and memory, designing algorithms for the search and generation of music motifs, developing analysis methods on EEG signals to decode musical minds, inferring public perception about e-cigarettes using large language models, and developing technologies to improve communication for deaf and hard of hearing children. The major objective of the REU is to encourage students to enter STEM graduate programs, but many of the projects will also lead to novel and publishable research in machine learning, music cognition, human computer interaction, and computer vision. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.