- By LaVonne Roberts
- Illustration by Joi Fulton
In 1985 when Walter Greenleaf received his PhD in neuroscience from Stanford University, he did something unimaginable. Rather than building or joining a prestigious research lab like his esteemed colleagues, he joined his friend Jaron Lanier’s startup, convinced that an emerging technology—virtual reality—could prompt a renaissance in medicine. He received a lot of pushback in his academic circles. “It was like I was leaving the church,” says Greenleaf, laughing about it today. “People thought I was a heretic.”
Greenleaf has devoted 40 years of his career to advancing the use of virtual reality in health care, straddling three overlapping but often radically different ecosystems—academia, tech, and medical devices. He is currently leading the development of clinical virtual reality (VR) products as a visiting scholar at Jeremy Bailenson’s Virtual Human Interaction Lab at Stanford, focusing on major human health problems like post-traumatic stress disorder, anxiety, depression, traumatic brain injuries, strokes, addictions, and autism.
Though many of his colleagues refer to him as the “godfather” of therapeutic virtual reality, he’s uncomfortable with the attention. “Call me Walter,” he says, which is part of his disarming charm. His ability to distill complex technology and academic jargon has everyone from NASA to major health systems, the largest U.S. health insurer,, and startup founders seeking his counsel. His deep medical knowledge and long history in VR is utterly unique.
The future
We’re in a period of rapid growth in virtual reality, augmented reality, and mixed reality technologies, collectively known as XR (for extended reality). Analysts suggest thatimprovements to VR hardware, such as smaller devices with less latency, will support its increased adoption by consumers and many different industries. Health care, workforce development, and manufacturing are all expected to be among the sectors most affected and advanced. For instance, virtual reality in the health care sector was valued at $2.08 billion in 2020 and is projected to reach $42.84 billion by 2028.
Much of that growth will be due to the simple fact that VR devices are increasingly interactive and affordable, and many now have integrated sensors. They can collect that sensor data for users or doctors, giving them a more comprehensive window into the everyday metrics of people’s health. “We have voice analytics capable of scoring tone as a biomarker for anxiety and depression that can also offer us clues about pain,” says Greenleaf. “Voice, movement, facial expressions, psychophysiological data like heart rate, variability, and EEG, gaze direction—all this data is going to help us do so much more than we’ve been able to do before.”
While health care applications of VR have become a growing trend among emerging technologies in recent years, Greenleaf has been a pioneer from the very beginning. “What’s interesting is Walter sees how much potential there is using these technologies not just to redefine care, but to redefine human interactions with the health care system,” says Brendon Hale, the chief research scientist leading the digital therapeutics research team at UnitedHealth Group, the world’s largest health insurance company.
A complicated patent saga
Greenleaf hasn’t just nurtured and steered the virtual reality ecosystem—he also actively sought to protect some of its key patents.
Virtual reality dates back to the 1960s when Ivan Sutherland developed the first computer-aided design system. Shortly after that, the federal government adopted the technology for war-game and flight-simulation purposes.
In 1984, Jaron Lanier founded the first company to provide general-purpose VR hardware and software: VPL Research. Lanier started in his Palo Alto garage with money he made from programming an early Atari video game. Barefoot, dreadlocked, bearded, and playful, he hosted celebrities like Yoko Ono, Stephen Spielberg, and the Grateful Dead, who were intrigued by the idea of mixing music and computer-generated objects. Jaron was highly effective at describing compelling scenarios of multiple possibilities the technology offered, setting off a media obsession with VR that could be summed up by what Wired Magazine’s Jaron Watch section called “forelash,” (the opposite of backlash) describing, “Negative or indifferent reactions to nonexistent, but already overhyped technologies.”
VPL collaborated with Mattel for a while, but cash flow limited product development. In the early 1990s, VPL Research partnered with Thomson-CSF, a military contractor partly owned by the French government. Strapped for cash, Lanier put the company’s entire intellectual property portfolio up as collateral on some small bridge loans from Thomson. By the mid-90s, with the advent of the internet, the world became obsessed with web surfing, and VR faded from the limelight.
In 1993, VPL Research was forced into bankruptcy by its largest shareholder in what appeared to be an attempt to grab ownership of the patent rights. However, five years earlier, Lanier had licensed medical rights to VPL’s patents to his friend, Walter, who refused to let Lanier’s company and its portfolio of fundamental VR patents and intellectual property revert to the French military contractor Thomson-CSF. Instead, he spent years defending a series of virtual reality patents until he could initiate a deal for Sun Microsystems to acquire the patent portfolio. “It took a lot of time and costly legal fees, but I was able to put together a deal with Sun Microsystems to obtain the IP with the goal of having Sun work to move the field forward,” says Greenleaf. “Of course, twenty years later, the patents have all expired.”
By protecting access to the patents, Greenleaf didn’t just bail out the fledgling technology—he helped secure the future of virtual reality.
Virtual reality in medicine
There are currently more than 171 million VR users worldwide. Greenleaf predicts 70 million more users will adopt VR within six years. The COVID-19 pandemic has created a surge in demand for digital medicines like VR that can be delivered safely to patients in their own homes. The number of VR startups has increased by 14 percent in less than a year, and venture capital is following. Companies like Penumbra, OVR, and Los Angeles-based AppliedVR, who provide VR-based treatments for addressing chronic pain for more than 200 of the top health care provider organizations, are garnering attention. Venture capital is now looking to Greenleaf for guidance.
“For 40 years, Walter has been a driving force,” says Sophie Lamparter, founder of DART Labs, a San Francisco and Zurich-based investment firm. “He’s steadfast in his mission to connect everyone from academia to startups, corporations, venture capitalists, and public entities like the FDA.”
A 2019 study in the journal Frontiers in Psychiatry from researchers in Stockholm identifies the opportunity to build on decades of VR technology’s evidence-based efficacy in treating phobias, stress, and pain. VR could offer an effective platform for incorporating cognitive behavioral therapy to treat depression within an immersive environment. “If we can evoke a cognitive state,” says Greenleaf, “then we can teach people the skills to manage that cognitive state.”
“It frustrates me that the health care system only interacts with humans when they’re sick.”
United Healthcare’s Hale agrees. “The health care system is largely reactive and doesn’t have the ability to take into account someone’s entire medical situation. What’s so interesting about the VR and XR space is that we get an authentic understanding of the world you’re seeing and how you, as an individual, react to your world [in a way] not fully realized with 2D technologies,” he says.
Improving the quality of health care depends on having adequate availability of providers. With an aging U.S. population, population growth, and greater access to insurance following the Affordable Care Act (ACA), health care demand is growing. The Association of American Medical Colleges (AAMC) reports that by 2033 the United States will face a shortage of between 54,100 to 139,000 doctors due to all those factors as well as physician burnout. Greenleaf and Hale believe that virtual technologies that can transform access for millions of Americans will help address part of that physician shortage.
“It frustrates me that the health care system only interacts with humans when they’re sick,” says Hale. “However, what excites me is the opportunity to move health care beyond episodic interactions with patients to a more full-spectrum understanding of their health and their environments so that we can deliver personalized care.”
Sniffing out opportunity
Four years ago, in Burlington, Vermont, Aaron Wisniewski had been obsessing over an idea so simple, he questioned it himself. What if adding scent to VR could increase immersion?
Like many others in the field, he looked to Greenleaf for advice. “When I decided to start my business, I went out to Stanford to try the technology and to understand Walter’s vision on how VR was going to change health care radically,” says Wisniewski. “Not in a grand, dreamy way, but tactically. He explained where VR is already making a difference, who the players were on the technology side, who were driving outcomes on the industry side, and who the adopters are.”
Research shows that when the brain registers a scent, it activates the amygdala-hippocampal complex, where emotions, memories, and associations are processed. “Smell may be the oldest and most undervalued sense,” says Wisniewski, “but it holds the key to improved virtual immersion and presence.” Convinced of VR’s therapeutic value, he launched his company Olfactory Virtual Reality (OVR) in 2020, debuting hisArchitecture of Scent platform to incorporate smell into therapeutic VR treatments.
Like every entrepreneur, CEO, or scientist interacting with Greenleaf, Wisniewski loves talking about his friend, collaborator, and mentor. “In the VR solar system, Walter’s the sun. He gives light to all of us and keeps us elegantly orbiting and traveling in the right direction,” says Wisniewski. “He guides us, keeping us in the right proximity to each other without collisions.”
A lot has happened since Wisniewski made his first trip to see Greenleaf. OVR is building on their research, demonstrating that stimulating the olfactory system in virtual reality can trigger memory, cognition, and emotion. OVR’s scent platform releases water-based scents during VR experiences, delivered via cartridges. Knowing that scents can immediately alter mood, Ketamine One clinics across North America partnered with OVR to provide virtual reality olfactory treatments to deepen immersion, increase relaxation, and lower stress.
Albert “Skip” Rizzo, the director for medical virtual reality at the University of Southern California’s Institute for Creative Technologies, uses OVR’s technology to enhance Bravemind, a VR exposure therapy program for military veterans with post-traumatic stress disorder (PTSD), by adding memory-provoking scents, like blood, diesel fuel, smoke, body odor, urine, garbage, and Iraqi spices, amongst others. While most commonly associated with returning war veterans, PTSD can occur in people of all ages, ethnicities, nationalities, and cultures, and it affects approximately 3.5 percent of U.S. adults every year.
The United States Air University has also built VR training modules to treat PTSD using OVR’s Inhale, a mindfulness, meditation, and breathwork wellness platform. “One of the biggest challenges in the Air Force and within the Department of Defense is suicide,” says Andy Clayton, assistant professor of leadership for Air University, who is excited about the U.S. Air Force’s collaboration with OVR. “Finding ways to improve human performance for our Airmen is the key to tackling some of the military’s most difficult issues, such as suicide. Smell is the strongest sensory system we have connected to memory, which up until now has not been applied to human performance within virtual reality. We can now unlock the true potential of virtual reality with the use of smell in virtual reality.”
Accidental fall becomes game changer
When Greenleaf talks about the future of VR in health care applications, he is most excited about companies like Penumbra, for which he serves as an advisor. It is attempting to harness the power of VR to enhance the neuroplasticity of the brain.
In 2004, Penumbra CEO Adam Elsesser cofounded the $10 billion global health care company with his best friend from college, Arani Bose. The company got its start with a catheter for the brain called the Penumbra System that operates like a vacuum cleaner, dislodging clots from arteries.
In 2017, Elsesser visited Sixense, a company specializing in VR hardware and software. Amir Rubin started Sixense seven years ago with Marvel movie producer Avi Arad, famous for movies like Blade, X-Men, and Spider-Man. “Walter is a believer in VR with heart,” says Arad, grateful for Greenleaf’s guidance. “It is difficult to communicate the power of something you need to experience firsthand to understand,” says Arad. Elsesser agrees.
With no background in gaming and wearing a suit jacket, Elsesser felt out of place. Sixense’s team put a headset with an attached battery pack tethered to a computer on him. “In the middle of playing that game, I heard two voices urgently yelling above the soundtrack coming from the game,” says Elsesser. “Close your eyes, close your eyes!” The battery was slipping, and Sixense’s employees were worried some of the imagery would disappear. Wanting to know what he was missing, he kept his eyes open. The floor of the castle that he was standing on went bright white. “I got a little unbalanced because I felt like I was falling, and then I fell,” says Elsesser. “I physically fell to the ground in my suit in front of everyone.”
After his first VR experience, Elsesser asked himself how Penumbra could use that same technology.
“At that moment, after years with my cofounder, a neurologist, I realized that if you can trick my brain to have that physical reaction, we are sitting on the magic tool to trick other people’s brains into recovering function in different settings,” Elsesser says. “The fundamental principle of stroke rehab is based on tapping into neuroplasticity to rewire the brain. If you can do that in virtual reality, you can tap into that concept and accelerate that function.”
“Because of VR’s ability to immerse us in somebody else’s point of view, we can facilitate empathy.”
Penumbra and Sixense teamed up to build the Real Immersive System, a VR platform for healthcare applications. It involves a head-mounted VR headset equipped with sensors and electrical components that measure motion and direction in physical space. It received clearance from the U.S. Food and Drug Administration in March 2019. In January 2020, physicians and therapists began using it. The new VR system displays and tracks interactive rehabilitation exercises for stroke patients, which requires the movement of limbs to complete tasks.
When Elsesser and his partner started Penumbra, they believed they could help several hundred thousand people in the United States affected by strokes. When they expanded their products to other indications, they saw an opportunity to help almost a million people. “When you look at the number of people we can help using therapeutic VR across the entire spectrum—mental health issues, memory, managing pain, the physical rehab work, and beyond—you’re looking at tens of millions in the U.S. alone. Why wouldn’t we try to do that?”
Looking forward
“We can shift attitudes and behaviors if we can activate the power of the brain’s reward system. This allows us to activate neuroplasty changes and reinforce behaviors and learning,” says Greenleaf. “Because of VR’s ability to immerse us in somebody else’s point of view, we can facilitate empathy.”
One technique that was originally developed and studied at the Stanford Virtual Human Interaction Lab is the use of an avatar of one’s “future self.” In one study, experiencing your future self in VR reduced stereotypical attitudes towards the elderly and evenencouraged people to save more for retirement. “By reflecting the effect of changes like a good diet, better sleep, or avoiding nicotine or alcohol on an avatar that looks like us but has been age-progressed to appear as we will in our later stages of life, we can leverage the power of our brain’s mirror neuron systems to motivate us to make changes in our behavior,” says Greenleaf.
Similarly, when a patient is discharged from a hospital, rather than handing them a stack of take-home instructions, Greenleaf envisions a better scenario in which a clinician’s avatar can interact with, inform, and motivate their patients to make necessary lifestyle changes.
To Lanier, who’s equal parts visionary, artist, and nostalgic digital philosopher, Greenleaf’s impact on therapeutic VR isn’t surprising. “Back in the 80s, it was socially awkward in startup circles and unusual to see an academic, but Walter was the ultimate Silicon Valley cool cat.” Pausing, his expression turns serious. “The thing is—Walter is unquestionably the principal pioneer of all therapeutic applications of virtual reality.”
Hale agrees with Lanier. “What’s interesting is the reality that Walter has seen for decades,” says Hale. “We’re just now developing the technological capabilities to realize his vision. In many ways, the world is catching up with Walter’s mind.”
Quelle:
https://neo.life/2021/08/a-hero-behind-the-vr-headset/
Editor’s note: This story was updated on 9/3/21 to correct the attribution of a quote in the 28th paragraph.
