Doctors using augmented reality were able to show patients‘ inside their own bodies, revealing growing fibroids, an enlarged prostate, and more hidden conditions.
The BBC Two show Your Body Uncovered with Kate Garraway shows how future technology could blend with medical appointments, but how close are we to using augmented reality (AR) in our hospitals?
In the show, doctors and radiologists work together with digital artists to create 3D representations of patients‘ bodies. Then, the patient wears a Microsoft HoloLens to view their limbs and organs, while their doctor explains exactly why and how their condition is affecting them.
You might be more familiar with the use of AR in apps like Pokémon Go, where digital versions of the Pokémon characters are placed in your real world via your smartphone camera. And while both augmented and virtual reality are growing parts of the metaverse, we haven’t seen innovative uses of AR brought to our televisions, until now.
Dr Stephen Quinn, a gynaecologist at hospitals in the NHS Trust Imperial College, appears on Kate Garraway’s new show to help a patient, Hilda, with a condition causing her swollen abdomen. After taking careful scans of Hilda’s body, the team are able to show her the growths, called fibroids, that are behind her pain.
„I’ve spent a lot of my career looking at MRI scans of pelvises, and having these images is extremely helpful in clinic,“ said Quinn. „But using augmented reality just took that to a whole different level. It was fantastic being able to to fully visualise exactly what was going on in the pelvis ahead of the surgery to remove the fibroids.“
Unfortunately, the technology is a way off being available on the NHS, but Quinn said AR’s use could be commonplace within the next decade.
How does augmented reality work?
For the show, radiologists at Imperial hospitals provided artists with in-depth scans of each patient. Dr Dimitri Amiras, a musculoskeletal consultant radiologist at Imperial, also worked on Your Body Uncovered with Kate Garraway.
First, patients would undergo routine scans.
„In order to define what the organ is and where the pathology is, that’s all done by radiologists. We are the ones to identify it and look at the imaging techniques work out what is good tissue, what’s bad tissue,“ said Amiras.
„Then, once we’ve got those images with relevant bits identified, digital artists may draw around them or even use artificial intelligenceto make all the pretty pictures and the shiny stuff.“
Once finished, the patients and doctors would wear an AR device to ’see‘ the body part in front of them. Each was 3D, and could be zoomed in or out, rotated, and compared to the same areas on a healthy individual.
One patient on the show, Hilda, is shown a representation of her uterus using real scans taken of the fibroids growing in her body © BBC / Remarkable TV
„The thing that’s exciting for me is that a lot of people don’t know a radiologist’s job is,“ said Amiras. „It’s nice for people to get excited about it and by the imaging. That’s what I do day to day – I look at the insides of people using imaging. It’s fascinating for me, and I think the rest the public would find it fascinating if there was more TV coverage of it. This is probably the first programme that has featured imaging primarily.“
When will GPs be able to use augmented reality in appointments?
While this technology is a way off being used in everyday life, Quinn says it is on the horizon.
„[Augmented reality] technology is available now, and we know things move on very quickly. I would imagine over the next 5 to 10 years, it’s likely that this will become more widely available. We know AR on smartphones is becoming increasingly popular.
„I imagine that we would be able to use this much more reliably when we get to a point in the future where people have more access to things like Google Glass or other sorts of smart ocular technology.“
However, one limitation will be the creation of the AR objects.
„It was very time-intensive, setting up the models and making it work,“ said Amiras. „While, in terms of the hardware, it probably was not the most expensive part, particularly in terms of health care stuff. It’s more the hours spent converting the images into something that’s relatable and easy to use.“
Patient Melina, Dr Purkayasthar and Kate Garraway use augmented reality headsets to look at a 3D representation of Melina’s skeleton © BBC / Remarkable TV
Universities are making use of the technology to teach medical students to perform complex surgeries, without the need for a patient to practise on.
„A few years ago we used AR for guidance in surgery. I was working in plastic surgery, in the reconstructive team,“ said Amiras.
„We’re doing a study with the breast team now, trying to see AR will help with breast reconstruction. And then the other thing we have done is to help student radiologists practise using a CT scanner for biopsies. [Without AR] you would need a CT scanner, and they are several hundred thousands of pounds. You would also need someone to run that CT scanner, and you would be potentially irradiating the person practising [because the scan uses radiation].
„So, we developed a way of doing that using the Microsoft HoloLens, and it would help people get familiar before they actually got to do it on patients.“
A patient, Rozi, was shown a 3D augmented reality version of her own lungs © BBC / Remarkable TV
Of course, the technology wouldn’t work for every doctor’s appointment.
„It’s difficult to say which conditions it will lend itself to,“ said Amiras. „Obviously, there are conditions that are more difficult to image. I think it will depend on the patient. Some will be very interested in understanding it in 3D, some patients will be happy with a verbal explanation or just an information sheet.“
Quinn said that it’s important to remember not everyone will want to see their insides.
„I always say to my patients, ‚Are you happy to look at the images together?‘ And, anecdotally, 9 out of 10 patients will be fascinated by their pictures and want to see what’s going on. But there are about 1 in 10 who don’t want to see,“ said Quinn.
„That makes it more challenging, and you have to then approach it in a different way, because it’s still important that patients have fully consented before they have any type of surgery. But we can use pictures, we can we can draw diagrams.
„The vast majority really want as much information as possible. That’s the age we live in now, people want to be as well-educated as they can be before going into any any type of procedure.“
How can I see my body in augmented reality?
While the AR on the show was specially commissioned, Amiras said patients can see 3D representations of body parts on The Visible Human Project.
„I think it started when a death row inmate in the US donated his body to science. They scanned him after he passed away, and also did a histology – so they cut lots of slices [of the body] and then turned it into 3D objects.“
- Watch Your Body Uncovered with Kate Garraway on BBC iPlayer
Quelle:
