Virtual operating room
In cooperation with partners from industry and research, Fraunhofer IIS has developed the surgical simulator HandsOn.surgery.
HandsOn.surgery allows the surgeon to practice procedures that involve drilling into the bone. Using a haptic arm as an input device and looking at a 3D scene displayed on an auto-stereoscopic monitor (does not require glasses) the surgeon controls a virtual drill and operates on a 3D bone model extracted from real CT-data. The current focus lies on procedures from the Ear-Nose-Throat (ENT) domain. An important use case here is to drill into the temporal bone (behind the ear) and create an access to the hearing channel.
Surgical training simulators are especially useful to young surgeons in order to get a feel for the forces involved in the interaction of the drill with bone tissue. The system has been validated by a group of experienced ENT surgeons who tuned the underlying physical model until it felt like the “real thing”. Risk structures such as nerves or blood vessels can optionally be overlaid on top of the 3D model. Additionally, the bone area removed by an expert surgeon can be made visible as a reference.
Force feedback, original sound recordings, and the autostereoscopic 3D monitor recreate the operation visually, haptically, and acoustically. An additional touchscreen is used to control the simulator, select among various drilling heads and simultaneously view where the drill’s tip is currently located within the CT-data from three directions. Recently at MEDICA 2018, interested visitors were able to see the system's functionality for themselves, and experts evaluated the simulator as part of the 19th Erlangen Petrosal Course. The next chance to experience HandsOn.surgery will be at the German ENT-congress in Berlin in May.
The product was developed as part of the HaptiVisT project, in which Fraunhofer IIS and the following partners are involved:
• szenaris GmbH (project coordinator)
• Clinic and Polyclinic for Ear, Nose, and Throat Medicine at the University of Leipzig Medical Center
• Clinic and Polyclinic for Trauma Surgery at the University Hospital Regensburg
• Ostbayerische Technische Hochschule Regensburg
• SeeFront GmbH
• Haption GmbH
The project is being funded by the German Federal Ministry of Education and Research (BMBF).