Todd Maddox examines how Extended Reality (XR) is an ideal tool to enable faster and more efficient familiarization with complex medical devices.

edical devices are essential for safe and effective prevention, diagnosis, treatment and rehabilitation of illness and disease. Many are used in the patient’s home allowing patients to lead normal lives, and to address their health needs on their own or with loved ones. Critically, high-quality education and training on these devices is required to avoid complications resulting from incorrect care and maintenance. For example, the Continuous Positive Airway Pressure (CPAP) system commonly used to treat obstructive sleep apnea, is a complex system that requires sterilization on a daily basis to avoid infection. In addition, poor cleaning and maintenance of a colostomy bag can be life-threatening. Finally, medical devices such as in-home peritoneal dialysis machines to address kidney disease, and insulin infusion pumps to treat diabetes are central to patient care, but these systems must be maintained to avoid complications. Although the existence of these medical devices increases the patient’s freedom and quality of life by allowing them to be treated at home, medical personnel are not readily available should something go wrong. This makes quality education and training critical to success.

By far the most common approach to medical device education is to have patients read documents describing the device and outlining the steps needed to care and maintain the device. Patients are expected to study and memorize the steps so that they can effectively care and maintain the device and avoid complications. This approach leaves it to the patient to determine proficiency and leaves the patient at risk when proficiency has not been achieved. With more complex systems, such as a peritoneal dialysis machine, document training will be supplemented with classroom and hands-on training with an expert. Although clearly superior to document training alone, this is time-consuming, costly, and is difficult to scale.

The neuroscience of learning and performance suggests that this traditional approach to medical device training is sub-optimal for at least two reasons. First, document training focuses almost exclusively on cognitive learning, when what is needed is broad-based experiential and behavioral learning. Second, hands-on training with an expert, although effective, is rarely extensive enough to ensure broad-based proficiency.

In this report, we discuss the neuroscience of learning and performance. We show why traditional approaches to medical device training are sub-optimal and can leave patients vulnerable to complication. We also show how immersive technologies like virtual reality (VR) and augmented reality (AR) hold great promise for optimizing medical device training.

As outlined in the figure below, the human brain is comprised of at least four distinct learning systems: experiential, cognitive, behavioral and emotional. As alluded to by Einstein, experience is at the heart of learning, and forms the foundation of VR and AR. The experiential system represents the sensory and perceptual aspects of an experience, whether visual, auditory, tactile or olfactory. Because every experience is unique and is immersive, this adds rich context and nuance to the learning. These enhance generalization and transfer of learning. The critical brain regions associated with experiential learning are the occipital lobes (sight), temporal lobes (sound), and parietal lobes (touch).

The cognitive system is the information system. This is the “everything else” that Einstein noted. The cognitive system processes and stores knowledge and facts that usually come in the form of text, graphics, or video. The cognitive system is truly amazing and is more developed in humans than in any other organism, but it relies on working memory and attention that are limited resources and form a bottleneck that slows learning. More information comes into the system and is available to the learner (the green arrows) than can be processed (the red arrow). This system encompasses the prefrontal cortex and hippocampus. Importantly, this system is slow to develop and starts to decline in middle age when health issues begin to arise.

The behavioral system in the brain has evolved to learn motor skills. It is one thing to memorize the steps needed to care and maintain a medical device, but it is completely different (and mediated by different systems in the brain) to know how to perform those steps. The behavioral learning system is fascinating, but its detailed processing characteristics are beyond the scope of this report. Suffice it to say that processing in this system is optimized when behavior is interactive and is followed in real-time (literally within milliseconds) by corrective feedback. Behaviors that are rewarded lead to dopamine release into the striatum that incrementally increases the likelihood of eliciting that behavior again in the same context. Behaviors that are punished do not lead to dopamine release into the striatum thus incrementally decreasing the likelihood of eliciting that behavior again in the same context.

The emotional learning system in the brain has evolved to facilitate the development of situational awareness — the ability to read nuance in a situation, to know what comes next, and to deal effectively with stress, pressure, and anxiety. This system affects processing in the cognitive and behavioral systems and helps patients learn to care and maintain medical devices under stress or pressure, or when “things just aren’t going right”. The critical brain regions are the amygdala and other limbic structures. Emotional learning is at the heart of situational awareness.

Using an insulin infusion pump as an example, let’s explore the traditional approach to training. The patient starts by reading documents that outline step-by-step how to care, maintain, and use the insulin pump. Then the patient may receive some hands-on training where they observe as an expert cleans and sets the pump up for usage. The patient may be given the opportunity to demonstrate the steps to the expert while receiving feedback. Because hands-on training is so time-intensive and costly, document and do-it-yourself training is usually emphasized.

From a neuroscience perspective, this approach starts by engaging the cognitive system as the patient reads documents and attempts to memorize the steps. Because the infusion pump is a 3-dimensional object and the system is dynamic (pushing insulin into the body), it is challenging to fully understand how the system works when the information being processed by the patient is 2-dimensional, static, and usually text-based. The patient must convert 2-dimensional static information into a 3-dimensional dynamic mental representation in the brain that accurately reflects the operation of the infusion pump. The cognitive effort needed to do this is enormous. Given the fact that working memory and attention are limited capacity resources, this process with be slow, challenging, and error-prone.

During the hands-on training, the patient can “experience” the system in action while watching an expert. This combines experiential learning with cognitive learning. When the patient is given a chance to care and maintain the device while receiving corrective feedback from the expert, behavioral learning is added to the mix. Although effective, behavioral learning is gradual and incremental and proficiency takes extensive practice, usually more than is provided in traditional training settings. In addition, it is rare that uncommon situations are trained.

Because XR training tools are available 24/7 the patient can have unlimited practice, and can train and test themselves under a broad range of environmental conditions Click To Tweet

At some point, the patient is deemed “ready” and they are allowed to take the infusion pump home. It is at this stage when the patient is at the greatest risk. They have a basic understanding of the care and maintenance of the infusion pump, but do not have extensive experience, and lack confidence. The brain tells us why. All of the training has been sequential and disjointed. Training starts with documents that train the cognitive system. Experience is then added to the mix, with behavioral training occurring last. What the patient really needs is for cognitive, experiential and behavioral learning to occur simultaneously, and in the environment where the infusion pump will be used (i.e., in the home). In addition, they need situational awareness training to deal with cases in which they are under stress or pressure, or when things are not going according to plan.

Now consider an immersive approach to infusion pump training. Suppose the patient dons a VR headset and is immersed in a 360 experience with an expert who has used an infusion pump for years. The patient can watch as the expert demonstrates and verbally describes the step-by-step procedures needed to care and maintain the infusion pump. The patient can view this from a third-person perspective, but also from the first-person perspective. The expert can demonstrate common pitfalls and how to address them, while describing them verbally. The expert can do all of this while demonstrating a calm demeanor. The patient can view the VR experience and can complete knowledge checks at the end to ensure that they have the requisite knowledge. They can view the experience as many times as they like until they are confident in their own skills.

To train the behavioral aspects of infusion pump care and maintenance, one could utilize an interactive VR setup, or an AR solution. Imagine using your tablet or a hands-free AR device that uses computer vision to identify the make and model of your infusion pump. The AR system uses visual assets such as arrows, highlights, text and such to guide you step-by-step through the care and maintenance of the system. As each step is completed, you are rewarded with success and move on to the next step in the process. You can practice the process as many times as you like, and even under time pressure.

From a neuroscience perspective, this immersive VR/AR approach engages all four learning systems in synchrony. In AR, assets are providing the necessary cognitive information. The learning is happening in real-time and through experience, with the patient generating the relevant behaviors and being rewarded or punished. This broad-based neural activation leads to a highly interconnected, context-rich set of learning and memory traces. These highly interconnected memory traces will be slower to decay over time leading to better long-term retention. Because these VR and AR training tools are available 24/7 the patient can have unlimited practice, and can train and test themselves under a broad range of environmental conditions. If they are unsure of a step, they simply pull out their VR headset or tablet, start the VR or AR software and are guided through the relevant steps.

Gone will be the days where medical device training for patients is inadequate, and patients are at risk of complication. With immersive technologies, patients can be given standardized and highly effective training from day 1. They can obtain experiential familiarization and practice as many times as they like. They can receive training on rare, but life-threatening situations so that they are prepared for anything. Because these immersive technologies broadly engage multiple learning systems in the brain in synchrony, the patient will obtain a strong knowledge base and behavioral repertoire that have been honed and testing. Patients will be more satisfied and confident, and fewer complications will arise.

Quelle:

https://medium.com/edtech-trends/report-xr-medical-device-training-6fad1cfb50b4

from Heather Andrew, CEO, Neuro-Insight

Key neurological insights from Mindshare UK’s groundbreaking report, ‘Layered’

Over the past few months, Neuro-Insight has been working in partnership with Mindshare UK and Zappar on ‘Layered’ – a first-of-its-kind study into the consumer, neurological and brand impact of augmented reality.

Until now, there has been little research carried out to understand the neurological effects of augmented reality (AR) and the ways in which the brain responds to various AR tasks and experiences. With ‘Layered’, we wanted to bridge this gap, unearthing AR’s true value – not just as a marketing and communications channel, but also an everyday utility that has the potential to transform the world around us, making it more rich, meaningful and engaging.

In this post I wanted to build on Jeremy Pounder, Director, Mindshare Futures’ blog discussing the future of AR and its implications for brands and share some of the key neurological insights from the UK’s first ever neurological study into the effects of AR on the brain.

Using neuroscience to study the effects of AR on the brain

To fully understand the effects of AR on the brain we recruited 151 UK smartphone users aged between 18 and 65. We allocated them into two cells, each with around 75 respondents, closely matched by their demographic, attitudinal and behavioral characteristics.

We used Steady State Topography (SST) brain-imaging technology to measure how the brain responded to different types of stimulus. STT measures the electrical activity in the brain in order to report (second-by-second) on a number of cognitive functions, including attention, personal relevance, emotional response and memory encoding. We can’t mind read (yet), but the technology can measure and quantify some very interesting things going on in your brain.

In order to capture these changes in brain response, we created six AR and non-AR tasks for the participants to complete with the aim of measuring how the brain reacts to augmented reality.

These six tasks were:

1. Google Translate: AR version vs. the online tool with text input
2. Product packaging: Zappar’s AR-enabled product packaging vs. traditional non-AR pack
3. Stacks (the game): Playing the AR version vs. the standard version
4. Ikea AR app: Placing virtual furniture in a room vs. browsing the Ikea website
5. Specsavers app: AR app vs. normal browsing on the website
6. BBC Civilisations website: AR experience vs. standard web experience

Upon arrival, respondents were sorted into the two matched cells and briefed on each of the six tasks they would be carrying out during the study. Half the sample completed the six tasks in AR and the other half completed the non-AR version.

Participants were asked to carry out the tasks on an iPad as they would normally do at home, with the order of the tasks rotated. Respondents were individually filmed as they carried out the tasks, allowing us to match their second-by-second activities to their neuro data.

It was truly fascinating to see how the brain reacted to augmented reality for the very first time. Like with a lot of the studies we undertake here at Neuro-Insight we expect a degree of variance from one study condition to another, but with AR we saw an unprecedented level of difference between the two cells of respondents in the study.

Three ways augmented reality affects the brain

As I’ll highlight below, augmented reality affects the brain in new and exciting ways. If you have the time, you should discover the full neurological findings in ‘Layered’.

Here are just three ways augmented affects the brain:

1. AR drives high levels of visual attention in the brain (almost double that of non-AR tasks)
2. Right now, AR elicits a ‘surprise’ response in the brain
3. What is stored, or encoded into memory is 70% higher for AR experiences

Let’s dive a bit deeper into these three core learnings to understand what’s happening in the brain when we talk about attention, surprise and knowledge retention.

1. AR drives higher levels of visual attention in the brain

Attention is a necessary precursor to any sort of brain response to communication – if people don’t register a brand message it can’t possibly have any sort of lasting impact. So, whether it be online, TV, billboards, or in our case, augmented reality, capturing attention is something that successful brands or business spend a lot of time, money and resource getting right.

One of the incredible findings from ‘Layered’ was AR’s ability to drive high levels of visual attention and engagement compared to non-AR tasks. In fact, upon completion of our research, we found that AR drove higher levels of attention that pretty much any other medium we have studied.

As you can see from the AR experience below, we see much higher levels of cognitive activity when the brain is exposed to the AR task compared to that of the non-AR task.

AR experience in ‚Layered‘ showing how augmented reality affects the brain

This wasn’t just a single experience either. Across the series of cognitive function measures carried out as part of the study, AR delivered almost double (1.9 times) the levels of visual attention compared to their non-AR equivalent.

This was particularly true of right brain visual attention, (attention to the overall feel or something rather than the detail). This is incredibly important when we talk about memory encoding, as there’s a direct correlation between the emotional intensity of a stimulus and how it’s encoded by the brain.

This finding, in particular, is crucial in showing AR’s ability to generate a more powerful response than equivalent non-AR experiences.

Cognitive activity during tasks. Brain activity measured using SST headsets; unit of measurement is radians, which equates to strength of brain response.

If we break this down further we can see increased levels of emotional intensity and visual attention in the brain compared to both TV and online viewing. What’s happening here is the brain is working much harder where we would expect it to. In particular, we saw incredibly strong levels of visual attention amongst younger smartphone users.

If we specifically look at levels of attention elicited by AR vs. TV, we see AR delivers a 45% higher level of attention. So, for media and brands competing to achieve cut-through, this is hugely significant in determining how to spread investment across channels.

So why are high levels of emotional intensity important? Simply put, emotional intensity is crucial when it comes to how we process experience because emotional intensity both drives and colors what the brain stores, or encodes, into long-term memory. This is highly significant for brands wanting to make a lasting impression on target audiences because memory encoding has a strong correlation with decision-making and purchase behavior.

These high levels of emotional intensity and attention were typified for us by the AR-enabled packaging concept W-in-a-Box created by Zappar for SIG. This was conclusive in demonstrating the ability of augmented packaging to increase levels of engagement, emotional intensity and attention.

Zappar’s augmented reality packaging concept for SIG.

2. Right now, AR elicits a ‘surprise’ response in the brain

The second insight found in ‘Layered’ came out of AR’s ability to surprise the end user. In neurological terms, what we saw in the study were lower measures of ‘approach/withdrawal’ (which captures the extent to which the user wants to move towards or away from a stimulus) in participants when undertaking AR tasks.

Average levels of brain response during AR and non-AR tasks.

A withdrawal response can be indicative of a number of different emotional take-outs; in this context, having also asked people about their conscious responses to the tasks in the study, it’s apparent that the “step back” response was a strong indication of the sense of surprise that occurs in the brain when smartphone users start an AR experience. Essentially what we’re seeing here is something that people aren’t expecting, a bit like a jack in the box. Your first reaction is to step back until you understand what you’re seeing.

It’s interesting to note that amongst men there was a stronger approach response than amongst women suggesting that, in this case, they were faster to embrace and engage with the unfamiliar.

Zappar’s AR-enabled gin bottle for Shazam and Bombay Sapphire

What does this mean for the future of AR? The capacity for AR to deliver surprising and emotionally powerful experiences is likely to endure for the foreseeable future as the software used to create AR experiences continually improves, enabling creatives to build more compelling and immersive AR experiences.

However, there’s also a bigger opportunity for AR to grow into more of a wider utility as it permeates our everyday lives. The qualitative research carried out in ‘Layered’, alongside our neurological findings, points to the fact that smartphone users welcome the technology’s ability to problem-solve different contexts and occasions of everyday life.

3. Memory encoding is 70% higher with AR

In neurological terms, if any type of branding or communication is to be effective, it needs to be encoded into long-term memory – otherwise, it will have little to no impact on any of our future actions.

Let’s put this into perspective. How many ads do you remember seeing yesterday? Even though you probably saw hundreds of ads across mobile, online and out of home billboards, if you’re like most people, you wouldn’t be able to recall more than a couple. (I’m able to recall exactly zero ads from yesterday!).

Graph showing left and right brain memory response to AR and non-AR tasks.

What we found in ‘Layered’ was that memory encoding was 70% higher in the AR tasks compared to the non-AR tasks. What this means is that AR can be a particularly powerful way to deliver information that is subsequently retained. Despite seeing higher visual attention for younger people, and stronger approach for men, the memory encoding response was similarly high for all groups of respondents.

What does this mean for brands? Simply put, the future for AR is looking promising. If advertisers can achieve 70% higher levels of memory encoding, then AR becomes a very interesting medium to put marketing budget behind to secure wider, more engaged reach.

Pernod Ricard collaborated with Shazam, OLIVER and Zappar to create an augmented reality experience for the box for their limited edition The Glenlivet Code product.

Neuroscience and the future of AR

As with most new mediums, the novelty factor associated with AR is likely to diminish to some extent over time. I’m sure we all remember the excitement of getting a broadband (or dial-up) internet connection and the seemingly endless possibilities it brought.

With this in mind, the withdrawal effect (jack in a box) I mentioned earlier will more than likely decrease as we see AR segway into an everyday utility that, much like the internet, becomes an integral part of our everyday lives.

It will be at this point that novelty transcends into strategy, and narrative and content become a much more integral part of delivering the AR experiences of the future. The ability to tell compelling and engaging stories will become increasingly more important as people’s brains come to expect everyday surfaces, products and packaging to be augmented.

Kate Spade utilise AR for the launch of their flagship store in Paris.

Conclusion

So what does all this mean for brands and how can they start leveraging AR as a more effective way of generating brand awareness in terms of attention, emotion and memory encoding?

These findings have huge implications on the way brands should be thinking about their long-term commercial strategies. As Neuro-Insight’s research certainly suggests, AR experiences are considerably more engaging and memorable than non-AR experiences, which presents a huge opportunity for brands to lead the way in leveraging the technology.

The folks at Zappar have seen AR move into the mainstream over the last year with some of the titans of tech, (the Googles and Apples of the world) investing in augmented reality, and now with ‘Layered’, we’re seeing the science to validate this investment. These findings don’t just stop at a brand level either. The implications are much more far reaching. For example, what will this mean for L&D professionals knowing that AR can drive greater levels of attention and engagement, but also enable staff to memorize more of their training? I know what I’d do.

Likewise, CPG brands can finally get ‘stand-out’ with their packaging on supermarket shelves whilst educating and entertaining their customers in the process. Now is the time to start thinking about your long-term AR strategy.

Don’t forget, you can take a deeper look into the neuroscience research carried out by Neuro-Insight in Mindshare’s ‘Layered’ report.

Project film for Mindshare’s Layered report.

Quelle:

https://www-zappar-com.cdn.ampproject.org/c/s/www.zappar.com/blog/amp/how-augmented-reality-affects-brain/