Cortical plasticity during reinforcement learning

Reinforcement learning (RL) is a fundamental learning process that is common across species and essential for cognitive flexibility and survival. In addition to neuroscience and psychology, RL also has proved valuable in the field of artificial intelligence (AI), achieving super-human performance in complex tasks. Understanding of the neural mechanism of RL would facilitate not only the development of diagnosis and treatment for learning and cognitive disorders but also development of novel deep RL architectures in AI. In this proposal, we aim to gain insights into how different brain areas work together to support RL. To tackle this problem, we use a behavioral task that entails two layers of RL at different timescales: fast RL to solve the task within a session and slow RL to learn the fast RL strategy over weeks/months of training. By leveraging mouse genetics and cutting-edge technology such as longitudinal two-photon calcium imaging of neural population activity and dopamine signaling, optogenetics, plasticity perturbation, and modeling with artificial deep RL networks, we will investigate the neural mechanisms of the fast and slow. In particular, we hypothesize that synaptic plasticity in the orbitofrontal cortex (OFC) plays a central role in slow RL. We will investigate the functions of OFC and its dopamine signaling in RL in Aims 1 and 2. In Aim 3, we will examine how OFC interacts with other cortical areas in RL. These aims will uncover the large-scale circuit basis of different aspects of RL and offer a novel conceptual framework to understand how RL is implemented in the brain.