German company ApoQlar is developing the Virtual Surgery Intelligence software tool, which uses mixed reality to help surgeons prepare for and carry out surgery. Abi Millar speaks to the developers about its potential applications in surgical procedures.
Could augmented reality (AR) be the next big thing within healthcare? Over the last few years, there has been a wave of startups in the field, focusing on everything from neuro-assistance for autistic people (Brain Power) to easier blood transfusions (AccuVein). Perhaps most excitingly, it could change the face of surgery, replacing many of the tools necessary today with a single, wearable piece of equipment.
These have included Medsights Tech, which uses image-reconstructing technology to give surgeons ‘X-ray vision’, and EchoPixel, which offers advanced medical visualisation software. Most recently Touch Surgery, a London start-up developing holographic surgery headsets, raised $20m from the backers of the Oculus headset.
“New applications – some we can’t even imagine yet – will help transform surgery and the surgical experience,” said Grossman, presciently, in a 2013 interview.
“It displays medical images in a 3D hologram, which the user can freely manipulate round the physical environment,” says Jakub Wlasny, lead developer at ApoQlar. “Those images are automatically fixed onto the body of the patient. This means doctors can slice through those holograms in order to explore the patient’s internal structures, without having to look at any specific devices or screens.”
After extensive testing, VSI Surgery has now been introduced to a hospital in Hamburg, where surgeons in the head and neck department are using it as a postoperative and preoperative assistant.
“We have found [the tool] gives us a much deeper and more detailed insight into the anatomical structures,” said Dr Hans von Lücken, senior physician. “But also for our interns the VSI is a big help. This way, they can orientate themselves more quickly and the more experienced doctor can better communicate their surgical approaches and procedures.”
Augmented reality technology involves ‘augmenting’ the real-world environment with various pieces of computer-generated input (sound, images or information), by way of a screen or headset. Although it is related to virtual reality (VR), there is one key difference: VR places the user in a completely immersive world, detached from reality, while AR overlays information onto the physical setting.
Although AR is only beginning to make its way into the healthcare space, its potential is significant. One key advantage is that practitioners don’t need to take their focus off the patient. Currently, surgeons need to switch between various reference points – from the person on the operating table to the medical images and patient data displayed on various screens. With applications such as VSI, they can access all the information they need while the patient remains in their field of vision.
“We want to deliver a more friendly and immersive way of working with medical images, helping doctors navigate around the surgical site and orientate those anatomical structures. This makes it quicker for them to proceed with surgeries,” says Wlasny. “We also want to eliminate as many of the other tools that are now used as we can, and put everything in one simple device.”
Looking through the HoloLens
As with many other technologies in this field, the VSI uses Microsoft’s headset, HoloLens. This device, which creates blended environments based on mixed reality, enables a live broadcast of the surgery from the surgeon’s perspective, meaning less experienced doctors could receive remote assistance.
Having already been used in many industrial settings, the HoloLens is just now beginning to make waves in surgery. In December, a team of French surgeons used HoloLens to livestream what they called the “world’s first surgical intervention performed with a mixed reality collaborative platform”. And in January, a team at Imperial College London demonstrated how surgeons can use HoloLens headsets while conducting reconstructive lower limb surgery.
“The application of AR technology in the operating theatre has some really exciting possibilities,” said Jon Simmons, who led the Imperial College team. “It could help to simplify and improve the accuracy of some elements of reconstructive procedures.
“While the technology can’t replace the skill and experience of the clinical team, it could potentially help to reduce the time a patient spends under anaesthetic and reduce the margin for error. We hope that it will allow us to provide more tailored surgical solutions for individual patients.”
Further applications of AR technology in the medical field
Since the tool uses machine-learning algorithms, it will continuously learn and improve, recognising more tissue types and acquiring more knowledge with every operation.
“I hope that in the long-term it’ll be a full medical product presented in many different medical facilities, and we hope to deliver as many useful functionalities for different kinds of specialties as possible,” says Wlasny.
From ApoQlar’s standpoint, there is no doubt that augmented and mixed reality will revolutionise current surgical options.
As the company’s founder Sirko Pelzl recently told the Hamburg News: “It’s not a question of whether this will happen, but whether we will lead this revolution.”
This article appears in the May 2018 edition of Medical Technology magazine
Journalist portfolio 