Theoretical Studies of Contrastive Local Learning Networks

Nontechnical Summary Digital AI networks astound us every day with the complex behaviors they learn. But they live on computers and consume enormous amounts of power—a single AI chip in a training cluster uses 600 watts of power. Real neuron networks in brains, by contrast, are physical materials that learn highly complex behaviors, running in a human on a mere 20 watts. This project focuses on tunable matter—physical systems that, like brains, can learn highly complex and desired emergent behavior. The PI studies Contrastive Local Learning Networks, which learn without a computer to perform AI-type tasks. Unlike AI networks, these networks are physical systems—forms of matter—that must obey physical constraints. These eliminate the need for a processor by enabling decentralized learning from highly limited information but also lead to new complexities beyond those confronted in AI networks. They are well controlled and characterized. Tunable matter is fundamentally different from typical materials, where samples behave about the same whether they contain a million or a trillion atoms. By contrast, the brain of a bee, with a million neurons, is far less capable than the brain of a human, with 100 billion neurons. Contrastive Local Learning Networks provide a unique theoretical opportunity to gain insight into how tunable matter differs from conventional matter, and to understand how learning emerges in physical systems. As a result, the project has implications far beyond materials theory, reaching into biology and technology. These networks have the potential to require far less energy and manufacturing cost than current AI, potentially revolutionizing smart devices such as wearables or robots. The PI receives many invitations to speak. When she visits an institution to give a talk, she always asks to meet with early-career scientists as a group; this has led to a network of people who email her for advice and support. To further disseminate the ideas and results of this funding and stimulate scientific discussions, the PI aims to propose workshops to Aspen Institute, Kavli Institute for Theoretical Physics (KITP), National Institute for Theory and Mathematics in Biology (NITMB), and the Santa Fe Institute and to write reviews and perspectives on this frontier topic. Technical Summary Theoretical condensed matter physics traditionally studies systems with given, fixed interaction potentials to predict and understand emergent collective phenomena. This proposal studies tunable matter—systems with constituent particles whose interactions can be adjusted individually to attain desired emergent collective behavior. Neural networks and biological neuron networks are many-body systems that illustrate a key property of tunable matter—the ability to acquire highly complex collective functionality. While many biological systems can usefully be viewed as tunable matter, this proposal focuses on a new class of tunable systems, electrical Contrastive Local Learning Networks (CLLNs). Laboratory realizations of such networks learn to perform supervised learning of tasks such as regression and classification without a processor or memory. Such systems pose the challenge of understanding tunable matter. Unlike neural networks, CLLNs are physical systems—forms of matter—that must obey physical constraints. These constraints are a double-edged sword; they enable learning without a processor using local learning rules but also lead to complexities not found in neural networks. Unlike tunable biological matter, which is embedded in organisms and interacts with other systems within them and with the environment, CLLNs are well-controlled and well-characterized, with clear separation from their environment. This proposal seizes upon CLLNs as a theoretical opportunity to gain insight into how tunable matter differs from conventional condensed matter and to understand how learning—in the machine learning sense—emerges in physical systems as a collective phenomenon. The research project has three thrusts: (1) to study noise in new CLLNs for unsupervised learning and generative tasks; (2) to learn dynamical behavior in CLLNs with non-reciprocal interactions; and (3) to understand how functionality can increase in complexity with increasing system size. Developing and studying more contrastive local learning systems is critical to understanding tunable matter as a class of condensed matter systems. The work is a circle in a Venn diagram that overlaps with the statistical physics of AI and biological physics of function—it is distinct in its focus on nonliving physical systems but gives insight into the other two. The research also has implications far beyond materials theory and condensed matter physics in fields including electrical engineering, robotics, mechanical engineering, computer science, developmental biology and neuroscience. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.