I-Corps: Medication Adherence System

The broader impact/commercial potential of this I-Corps project is to enhance the capabilities of nurses and caregivers as well as anyone who needs to administer/verify the medications (injection and pill) and prevent potential human errors. It is very essential as medication administration errors are known to be detrimental to patient safety, compromise patients’ confidence in the healthcare system, increase patient’s healthcare costs, and adversely affect the patient’s quality of life. This will help society by promoting trust between patients, families, and healthcare providers. Moreover, the broader impact of this project goes beyond the healthcare providers, nurses, caregivers, nursing home nurses, and similar ones to the general public as taking wrong medication could occur to anyone. Furthermore, the tool would provide new data on the number of times mistakes were prevented for training of the medical staff as well as all general public on the possibility of such errors. This I-Corps project is based on the development of an artificial intelligence-enhanced medication administration software for Augmented Reality devices (including smartphone and Augmented Reality glasses) to enable mobile, accurate and automatic verification of medication at the point of care. Via this Augmented Reality application, patient identification and medication information retrieval are the first steps, where patient identification is verified through face or barcode recognition, and medication information is retrieved from the electronic health record system. The system verifies medication name and time through barcode scanning or neural network, and a custom-designed deep learning framework recognizes injection dose/volume. This system also provides heads-up instructions and reminders for injection routes, and automatically saves images and verified information into the records system for documentation purposes. To improve the system’s robustness to the adverse environment, such as imperfect viewpoint, ambient light/background, and low resolution of the object, a multi-stage image processing procedure is also incorporated by drawing inspiration from simple human intuitions. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.