CAREER: Investigating and Combating Micro Signal Attacks in Video Conferencing

Video conferencing applications have been broadly used to connect geographically distant people for work, school, and social interactions. Because they use audio and video, and these can show information about a participant's behavior and environment, people choose to turn off microphones or cameras when they have related privacy concerns. However, microphones and cameras may leak other kinds of information beyond the semantics of what is seen and heard, raising privacy risks people are unaware of. This project's goal is to examine those risks, looking at the many sensitive "micro signals" that are sent over the network through the legitimate visual and acoustic channels of video conferencing. These signals are too tiny for humans to recognize, but detectable by machines, and through careful signal processing they might inadvertently reveal information about people's location, off-camera behavior, interaction with their computer, and other things people might want to keep private. Through better understanding the risks and developing methods to mitigate them, this project will advance understanding of side channel attacks and increase online meeting privacy. The project also contributes to cyber security education through curriculum development, demo platform implementation, graduate/undergraduate student training, K-12 involvement, public outreach, and underrepresented student engagement in research. This project advances the knowledge of acoustic sensing and visual sensing and brings the security protection of video conferencing down to the micro signal level. It exploits the two-way audio channel of video conferencing to send malicious acoustic signals remotely, which sense the user's current physical surroundings and return to the attacker with location-specific echo signals. Deep learning algorithms are developed to circumvent the echo cancellation mechanisms enforced by audio streaming systems, maximizing the retrieval of sensitive echoes for attackers. The project further uncovers the user's on-screen inputs during video calls, which are out of the webcam's view and believed to be safe, including online voting choices, touchscreen device inputs, and physical keyboard typing. The user's eye motions, keystroke-induced camera vibrations, and the monitor's screen lights are exploited for privacy inference. To prevent privacy leakage from micro signals, this project develops both targeted defense approaches, which address each attack separately, and general micro-signal removal techniques based on compression and decompression, which can handle different micro-signal attacks and their variants. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.