Collaborative Research: EAGER: Enhancing Security and Privacy of Augmented Reality Mobile Applications through Software Behavior Analysis

Augmented Reality (AR) mobile apps mix virtual reality (VR) with reality to provide revolutionary user experience in tasks such as navigation, virtual meetings, exhibitions, gaming, and translation. To make virtual objects appear to be attached to real-world objects (e.g., surfaces such as walls and human faces), AR apps place virtual objects at a location (called anchors) relative to the real-world objects (called trackables) identified by the AR devices. However, AR apps’ privilege to retrieve and manipulate camera output from AR devices (i.e., often part or the whole of users’ eyesight), enabled by AR framework software APIs, result in unique security and privacy concerns, such as vandalism of VR arts and tracking bystanders. Existing defense mechanisms of AR apps (e.g., the permission system in an Android smartphone) do not model the unique behaviors of AR elements (e.g., trackables and anchors) and are too coarse grained to detect and mitigate potential privilege abuses of AR apps. The goal of the project is to (i) develop a novel software analysis framework that detects and mitigates VR app’s security and privacy risks, and (ii) conduct a large scale study on real AR apps (e.g. AR-assisted Driving and shared AR arts) to study their unique security issues. More specifically, the project will develop static and dynamic program analysis with a focus on the unique AR elements (e.g., trackables and anchors) to detect two major types of privilege abuses: abuses of read access to camera output and abuses of write abuses to screen. In particular, the project will develop (1) trackable-anchor analysis that formally models the software behaviors of AR elements and their life cycles in AR software; and (2) anomaly detection techniques for read and write abuses using anomaly detection models. The project will then conduct a study on a large number of real VR apps by applying the developed techniques to evaluate the effectiveness of the techniques and uncover unique security issues. The success of this project will lead to more secure AR apps and AR systems, and the study will deepen the understanding of the security risks and vulnerabilities in AR apps. The proposed research will also enable finer-grained AR access control on dynamically generated virtual objects. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.