Butterfly Network’s AR-assisted ultrasound can let doctors guide your own home scans.
My wife fell down the stairs over a month ago and we didn’t know if her foot was broken or not. We didn’t go to the doctor to find out. We’re not the only ones. So when I saw Butterfly Network’s Zoom demo of its handheld scanner — and how a doctor could look for possible fractures from afar — I thought: When can our house get one of these? It’s like ’s tricorder, but in the real world. That has been in motion in the tech world for years, but recent developments have brought it closer than ever.
Butterfly Network’s handheld ultrasound device, the WWDC and I’ve thought about it ever since. What grabbed me was the device’s promise of a future of remote patient assistance, using pop-up 3D instructions that could guide a patient to use the device properly, too., has been available since late 2018. Connected to a phone or tablet, it can act like a little portable medical tool. But it’s designated for „doctors or qualified practitioners only.“ I saw a demo last year at Apple’s
Recent FDA changes allow medical imaging tools to be used remotely by doctors in emergency situations. This means Butterfly Network’s tech could be used at home now, using remote AR-assistance software that taps into to render 3D directions as a person uses it.
I see the little hair trimmer-sized scanner point over someone’s body, while in another window, I see the pulsing echoes of the ultrasound. I also see arrows and markers, which can be placed there on a patient’s iPhone or iPad to remotely guide where to look, or to mark up areas of interest. Butterfly Networks‘ VP of Product, Matt de Jonge, thinks of it like a „magic mirror“ that helps someone orient the scanner and follow a doctor’s live guidance at the same time.
The Butterfly iQ’s capabilities include looking at pregnancies, possible broken bones or even looking at lung conditions: According to Butterfly Network, doctors are currently using the tech to look remotely for signs of lung damage associated with COVID-19.
The TeleGuidance feature works on more recent iPhones and iPads (phones, but not with the AR guidance yet. Butterfly Network’s de Jonge sees more advanced phones with better computer vision and room scanning (the new , or improving software efforts) as helping more down the road.and later, and more recent ) that have the power to handle the AR and also room-scanning, machine-learning guidance needed to track the ultrasound device’s position in relation to a human body. The handheld scanner also works with some Android
Butterfly Network is careful to make clear that right now, the FDA permission is temporary for the duration of theemergency. Per the company, „We are working closely with FDA to bring safe, effective, remote-guided ultrasound imaging to the medical and patient community. We are grateful for the opportunity to advance clinical care during this challenging time.“
The Expert View, where a doctor can send messages to guide the person using the ultrasound equipment and monitor results. Butterfly Network
I’ve used AR headsets like thethat have explored remote assistance and guidance via AR and to do the same. This is the first time I’ve seen this for an essential home need like telemedicine.
The ultrasound hardware is, to be clear, expensive: $2,000-plus subscription services, although the company says that doctors could prescribe them, or loan them. There’s a possibility of insurance coverage. That’s why I think it’ll be some time before I get one of these in my own home.
The idea of AR assistance in general, though, is something that could keep being useful in all sorts of ways. A technician guiding you through home repair, or technical support. A contractor trying to understand your home and offer advice. Doctors. This type of home assistance isn’t really here much yet, but it looks like a promising next step, maybe even after lockdown.
Correction, 8:49 a.m. PT: Clarifies that the Butterfly iQ can help doctors, specifically, look for signs of lung damage related to COVID-19.
Butterfly Network’s Teleguidance feature on an iPhone, with little arrows that direct using AR. A doctor can be somewhere else.