FET: Thermodynamic effects of network topologies in distributed computers

Every computing device -- from a smartphone to a data center to the human brain -- must consume energy to process information. Modern computers comprise a vast number of different computational subsystems. The same is true of the human brain. Moreover, the minimal energetic cost of a set of connected computers depends crucially on the specifics of how those computers are connected. This provides a refined version of the scientific question that applies to animal brains as well as the subsystems of individual computers: How precisely does the way a computing system’s parts are connected to one another affect the minimal energy cost to run that system? This project investigates those questions, using cutting edge nonequilibrium statistical physics techniques, some of which were pioneered at the Santa Fe Institute. The results will provide the scientific foundation needed to design far more energy-efficient computers, artificial intelligence hardware, and telecommunications networks -- reducing operating costs, strengthening U.S. competitiveness in computing technology, and contributing to national energy security. In addition, the project may provide fundamental new insights into the human brain – and specifically how it manages to operate with such a small energy cost. The project will also train researchers advancing the next generation of scientists in this emerging field. This project develops a theoretical and computational framework for quantifying the thermodynamic cost (i.e., energy dissipation) of distributed computing systems, with a focus on how the topology of the network connecting a computational system's components determines its thermodynamic cost. The central analytical tool is Mismatch Cost (MMC), a strictly positive lower bound on thermodynamic cost which was derived under prior NSF-funded work by the investigator. Crucially, MMC is hardware-agnostic - it applies universally across physical substrates, from conventional semiconductor circuits to neuromorphic hardware to biological neural tissue. Accordingly, results concerning MMC are guaranteed to always apply, no matter how technology evolves, and no matter whether we are considering biological or artificial computing systems. This makes MMC uniquely suited to studying the thermodynamic cost of distributed systems of computers. While it has now been established that MMC depends crucially on the topology of the network connecting the components of a computer, very little is known about the details of that dependence. To investigate this issue, we must have precise control over both the network topology and the computation being performed. To achieve such a control the project focuses on trained neural networks and restricted Boltzmann machines, as stylized models of distributed computers. These systems will be trained on well-defined computational tasks — including instances of the canonical KSAT benchmark problem in computer science — while their internal network connectivity is systematically varied. Using MMC, the project will then characterize how thermodynamic cost, computational speed, and robustness to component failure jointly depend on network structure. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.