Planning System and Augmented Reality
In this project, a system to intuitively visualize and plan the surgical intervention using the Minimally Invasive Robot-Assisted Computer-guided Laserosteotome is developed. For this purpose, medical imaging data are rendered in 3D in a virtual room in real-time. The groundbreaking technology behind our system is that no preprocessing of the medical imaging data as segmentation or alike is needed. The current system uses the HTC Vive glasses and is called SpectoVive. The imaging data objects can be scaled, moved, and cut in real-time, which requires the rendering 180 frames per second. Due to the three-dimensional depth perception, the planning of osteotomies (bone cuts) with such a system is much more precise and less time consuming compared to conventional intervention planning. Furthermore, organs at risk and other structures in vicinity of the cut can be recognized more easily and can be considered from the very beginning of the intervention. Besides serving as a planning tool, SpectoVive can be used for diagnostics and teaching purposes.
Surgery Planning in Virtual Reality at the University Hospital Basel.
In the future, we plan to use augmented reality techniques during the intervention to show medical imaging data and the robotic endoscope in action together with the undistracted view of the patient to the surgeon. This enables the surgeon to monitor the intervention and allows stepping in if necessary.
For this project, we were selected to be a part of the HTC Vive Tracker development program and were provided with one free HTC Vive Tracker before release.
Project leaders: Marek Zelechowski and Balázs Faludi
B. Faludi, N. Zentai, M. Zelechowski, A. Zam, G. Rauter, M. Griessen, P. C. Cattin (2021) Transfer-Function-Independent Acceleration Structure for Volume Rendering in Virtual Reality, High Performance Graphics (HPG) 2021 Symposium Papers read
M. Żelechowski, B. Faludi, G. Rauter, P. C. Cattin (2021) Volume Rendering-based Patient Registration for Extended Reality, International Workshop on New Trends in Medical and Service Robots (MESROB)
M. Żelechowski, M. Karnam, B. Faludi, N. Gerig, G. Rauter, P. C. Cattin (2021) Patient Positioning by Visualizing Surgical Robot Rotational Workspace in Augmented Reality, Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization read
P. M. Maloca,B. Faludi, M. Zelechowski, C. Jud, T. Vollmar, S. Hug, P. L. Müller, E. R. de Carvalho, J. Zarranz-Ventura, M. Reich, C. Lange, C. Egan, A. Tufail, P. W. Hasler, H. P. N. Scholl, P. C. Cattin (2020) Validation of virtual reality orbitometry bridges digital and physical worlds, Nature Scientific Reports 2020 read
B. Faludi, E. I. Zoller, N. Gerig, A. Zam, G. Rauter, P. C. Cattin (2019) Direct Visual and Haptic Volume Rendering of Medical Data Sets for an Immersive Exploration in Virtual Reality. In: Shen D. et al. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science, vol 11768. Springer, Cham. read
D. M. Croci, R. Guzman, C. Netzer, L. Mariani, S. Schaeren, P. C Cattin, G. F Jost (2020) Novel patient-specific 3D-virtual reality visualisation software (SpectoVR) for the planning of spine surgery: a case series of eight patients, British Medical Journal (BMJ) Innovations, June 2020, Volume 6, Issue 4 read
P. M. Maloca, J. E. R. de Carvalho, T. Heeren, P. W. Hasler, F. Mushtaq, M. Mon-Williams, H. P. N. Scholl, K. Balaskas, C. Egan, A. Tufail, L. Witthauer, P. C. Cattin (2018) High-Performance Virtual Reality Volume Rendering of Original Optical Coherence Tomography Point-Cloud Data Enhanced With Real-Time Ray Casting, Translational Vision Science and Technology (TVST) July 2018, Volume 7, Issue 4 read
No open projects currently