Trackerless Surgical Navigation for Dynamic Environments in Minimally Invasive Liver Surgery

Project Summary/Abstract The complex variations of the liver during surgical resection present a significant challenge for surgical navigation in minimally invasive liver surgery (MILS). This proposal aims to revolutionize MILS navigation by leveraging novel computer vision and deep learning algorithms to integrate multiple imaging modalities, including regular and near-infrared (NIR) laparoscopy, ultrasound, and MR/CT. Our first objective is to develop a pioneering dynamic simultaneous localization and mapping (SLAM) approach, enabling real-time tracking of liver surface variations from stereo laparoscopic videos. To mitigate the accumulative errors during lengthy procedures, we will utilize near-infrared tattoo technology to establish easily recognizable artificial landmarks. Our second objective is to develop an efficient 3D ultrasound volumetric reconstruction method and a comprehensive registration method between SLAM, ultrasound, and MR/CT data to improve the accuracy of localizing internal vessels and tumors, as well as minimize accumulative errors. Preliminary results have shown promise, validating the feasibility of our approach. Moving forward, we will integrate these algorithms into the 3D Slicer software to provide an intuitive interface to surgeons and validate our navigation system using both ex vivo and in vivo porcine livers. Our project benefits from a strong research team. Dr. Haoyin Zhou (PI) has extensive expertise in computer vision and deep learning, underpinning our innovative SLAM approaches. Dr. Jayender Jagadeesan (co-investigator) brings invaluable experience in surgical navigation, while Dr. Sandy Wells (co-investigator) is a renowned authority in ultrasound image registration and segmentation. Dr. Jiping Wang (Other Significant Contributor) contributes extensive expertise as an experienced general surgeon. Together, our team from Brigham and Women's Hospital, Harvard Medical School, is poised to make significant strides in advancing MILS navigation.