CAREER: Transient Execution Attacks and Mitigations Across Different CPU Platforms

Transient execution attacks represent a large class of serious computer security threats, where the attacker alters the intended execution of the code in for short periods of time, rather than permanently. The attackers may transiently execute arbitrary code, including instructions that are otherwise illegal. Such attacks have led to severe security breaches, including data leakage and breaking existing security mechanisms in hardware. There is an urgent need for efficient mitigation across different CPU platforms today and CPUs in the future. This project aims to develop an automatic evaluation framework and a set of efficient protection schemes for transient execution attacks across different CPU platforms. The project will first develop a novel reinforcement learning (RL)-assisted framework for agile attack surface analysis and mitigation evaluation across CPU platforms. With attack building blocks in the framework, the RL-assisted approach enables practical evaluation of attacks on black box processors without significant manual efforts, offering an effective tool to comprehensively evaluate existing and future mitigation strategies. Second, the project will explore commercial hardware security features to protect the integrity of transient execution efficiently. Existing research focuses on Intel hardware features, leaving many other hardware security features unexplored. This project defends the attack efficiently from a new angle by protecting pointer access during the transient execution. Third, the project will co-design block cipher and microarchitecture to offer a secure pointer protection mechanism against both transient execution attacks and memory safety attacks. The project will also integrate research with education and outreach by creating course modules to help bridge the gap between concepts and practice, engaging students at all levels with hands-on experiences. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.