FULLY AUTOMATED CONVERSION OF GLIOMA CLINICAL MRI SCANS INTO A 3D VIRTUAL REALITY MODEL FOR PRESURGICAL PLANNING

Medical images have a tremendous amount of spatial information for localizing tumors and planning surgical interventions. However, viewing data in 2D, even in a multiplanar viewer, does not convey the complex 3D relationships of the anatomy. This places the burden on clinicians to utilize visuospatial processing to generate their mental representation and further requires working memory during procedures to link 2D imaging to the 3D surgical field. In this project, we are developing an automated pipeline to build rich 3D virtual reality (VR) models from clinical MRI of glioma patients using deep learning. The current project uses structural and diffusion MRI to automatically create 3D VR models of gray matter, white matter, blood supply, tumor core, tumor, and white matter fiber tracts. The VR models can aid in surgical planning and generate a better understanding of the extent and arrangement of the tumor relative to other structures in the brain.