Tag

EEG

Study Suggests EEG Could be Used to Predict & Prevent VR Motion Sickness

A new study on VR motion sickness concludes that certain brain activity detectable by EEG strongly correlates with VR motion sickness. This finding suggests that it’s possible to quantitatively measure and potentially prevent VR motion sickness.

While virtual reality opens the door to incredible possibilities, what we can actually do with VR today is at least somewhat limited by comfort considerations. While developers have steadily invented new techniques to keep VR content comfortable, scientists continue to work to understand the nature of motion sickness itself.

A new study published by researchers at Germany’s University of Jena in the peer-reviewed journal, Frontiers in Human Neuroscience, intentionally induced VR motion sickness in participants while measuring brain activity.

14 subjects were hooked up to an EEG cap and donned a PSVR headset. In the headset the participants were exposed to increasing levels of artificial movement to induce VR motion sickness over the course of 45 minutes. In addition to recording brain activity via EEG, the subjects also subjectively rated their motion sickness symptoms throughout the experiment.

The researchers found a common pattern in the change in brain activity that closely corresponded with the subject’s own perception of motion sickness.

Specifically, the researchers write, “relative to a baseline EEG (in VR) the power spectrum for [brain] frequencies below 10Hz is increased in all brain regions. The increase in frequency power was correlated positively to the level of motion sickness. Subjects with the highest [perception of motion sickness] had the highest power gain in the theta, delta, and alpha frequencies.”

Researchers Matthias Nürnberger, Carsten Klingner, Otto W. Witte, and Stefan Brodoehl offer the following conclusion:

We have demonstrated that VR-induced motion sickness is associated with distinct changes in brain function and connectivity. Here, we proposed the mismatch of visual information in the absence of adequate vestibular stimulus as a major cause according to the model of predictive coding. […] Differentiation which changes in brain activity is due to the sensory conflict or caused by motion sickness should be investigated in further studies. Given the increasing importance of VR, a profound understanding of the constraints imposed by [VR motion sickness] will be increasingly important. Measures to counteract the occurrence of MS or assist in detecting it at an early stage will undoubtedly improve the progress with this promising technology.

The findings offer further evidence that motion sickness can be objectively detected through non-invasive hardware like scalp EEG, which could be used to guide future research into VR motion sickness and VR comfort techniques.

For one, such EEG measurements could be used to objectively evaluate the effectiveness of VR comfort techniques.

Presently developers of VR content employ a variety of well-known VR comfort techniques like snap-turning and teleportation to reduce the odds of VR motion sickness. But not all VR comfort techniques may be equally effective compared either to one another or when compared across individuals. Establishing a quantitative measurement of motion sickness via EEG could help improve VR comfort techniques or even discover new ones by providing clearer feedback while making testing more objective.

Such measurements could also inform comfort ratings as presented to end-users, to help those sensitive to VR motion sickness find appropriate content.

Further, EEG detection of motion sickness could potentially be used on a real-time basis to predict and prevent motion sickness.

EEG brain sensing technology is becoming increasingly accessible and has already been integrated into commercial VR hardware. In the future, headsets equipped with EEG could allow developers to detect a user’s level of motion sickness in real-time, allowing for content adjustments or for VR comfort techniques to kick in automatically to keep users comfortable while they play or work in VR.

Quelle:

Study Suggests EEG Could be Used to Predict & Prevent VR Motion Sickness

6. Dezember 2021 0 comment

Knowledge Research

Input Methods in XR: EEG/EMG

EEG sensors utilize brain waves, and EMG sensors use muscle response to manipulate digital objects in the virtual space. There aren’t very many options on the market right now, and are mostly only used for specialized cases such as medical rehabilitation or accessibility for motor function impairments. The options that are available right now are used on the higher-end 6-DoF headsets.

This technology is still experimental, and is not ready for mass consumer adoption.

EEG Pros

It frees your hands from the use of controllers.
It’s a good option for people with motor-function impairments.
It’s useful in measuring the state of mind of a person throughout the experience.

EEG Cons

It takes time to calibrate the sensor with each individual person before it can be used effectively.
They would be expensive for the majority of consumers.
These devices require more concentration than traditional input methods.

Society isn’t fully ready for mass adoption

It’s my belief that society will need time before it’s ready to adopt machine-brain interfaces due to the current lack of trust in the technology. It’s similar to how we had issues with adoption of the Google Glass many years ago. Society wasn’t quite ready for that invasion of privacy. Now, people are adopting Augmented and Mixed Reality at much higher rates. A similar shift will need to take place for this tech as well.

Once people become more familiar with how it works, and they see more application examples in the wild, they will be more willing to accept it. The same will be the case for brain implants, only with a much, much higher hurdle.

Connect 2021, EMG inputs

EMG Pros

It frees your hands from the use of controllers.
It could allow people who have lost appendages to regain use in virtual space depending on severity.
EMGs reduce or eliminate the need for full body suits to track your body movement.
You’ll be able to have more precise movements than traditional hand tracking – such as typing and writing – without the need for gloves.
It’s a more natural interaction for digital object manipulation.
Your hands don’t have to be within line of sight of cameras on your headset to be tracked like they do with camera-based hand tracking.

EMG Cons

The sizing of the EMG sensors can be an issue for large or small limbs. They may need multiple sizes as one-size-fits all options won’t support a large enough range of people.
They can be expensive.
They’re still experimental and can be buggy.

One comment about the Connect 2021 keynote

During the keynote presentation, Mark Zuckerberg makes a comment about using an EMG to interact just by thinking about it. It seems that he may be mixing up EMG sensors with EEG sensors since you do have to move your muscles to be able to pick up readings with an EMG.

Recommendations

At this stage of technology maturity, it is not recommended that EMG or EEG sensors be used as primary input methods for XR applications unless there is a specific educational, medical or rehabilitation use case. However, EEGs can be useful during the development and testing phase of the project creation process as a way to measure the effects the experience has on a participant at any given time.

The lack of availability of options to try means we don’t know enough about the technology to come up with good UX best practices at this time. That said, it can be safe to say you should also follow best practices for gestures when creating hand interactions with EMGs.

Learn more

Electromyography (EMG) by Johns Hopkins Medicine
The Introductory Guide to EEG (Electroencephalography) by Emotiv
Facebook, now Meta, shows off new neural interface tech during Connect conference by Russell Holly, CNET

Quelle:

Input Methods in XR: EEG/EMG

22. November 2021 0 comment

Knowledge Research

Training Your Brain To Relax On A Virtual Island

A trip on London’s underground is rarely a relaxing experience, but the Covid-19 pandemic has added an extra level of anxiety for many.

I’m off to try out a new technology that promises to train my brain to relax. Sitting far apart from other passengers in the carriage while wearing a hot surgical mask, I’m hoping it works.

I’m meeting Dr Jamil El-Imad who enjoyed a successful career in the computer industry before, in his words, „getting sucked into neuroscience“. The Lebanese-born IT expert knew all about computer languages and was intrigued by the similarities between data and the way the brain processes information.

Recognising the potential of virtual reality devices he worked on how to use cloud computing techniques to capture and analyse brain signals and create a machine to replicate the meditation experience.

Neuroplasticity, the adaptability of the brain, is the science here. Neurons, the circuits of the brain, become stronger the more they are exercised. They are changeable. And they are individual.

Neurons gripped Dr El-Imad. He sees our neurons as „a forest where every tree is different“. This means that each person needs their own approach to gaining mental resilience.

GETTY IMAGES – Neural pathways can be reinforced with training

His innovation was to integrate the technology of the electroencephalogram (EEG), which monitors electrical activity in the brain, with a virtual reality headset.

The combination means that the response of the subject’s brain to images can be measured.

The biofeedback can be analysed instantly using the affordable computer power offered by cloud computing, industrial levels of data processing rented over the internet.

Murali Doraiswamy, professor of psychiatry and behavioural sciences at Duke University School of Medicine in North Carolina in the US and a former adviser to Dr El-Imad’s company NeuroPro, says that meditation and the whole realm of mindfulness are proven treatments for some conditions.

„There are many types of mindfulness and it’s not a panacea. But in the last 30 years mindfulness has gone from fringe science to the mainstream. It can be as effective as medication in preventing a recurrence of depression. It won’t be for everyone, but it can change our outlook.“

NEUROPRO – The headset is designed to train the user’s attention

The first contact I get with this system, called the Dream Machine, is when the EEG headset is slipped over my forehead. It’s a semi-circular band containing sensors that transmit wireless signals revealing just what is going on in my brain.

The EEG headset takes feeds from my level of attention and will register when my mind veers off course. Over multiple sessions this should help subjects to master their own mind and gain more control over their thoughts, which is a definition of mindfulness.

The VR headset is not a bulky helmet. In fact there’s no sense of the weight due to a springy coil that holds it and takes the weight off your head. Without that weight there’s no sense of enclosure either.

Hearing ethereal pipe music, I see an island floating in space. Waves lap on a beach and beyond giant Easter Island statues are planted among the rocks and palm trees. White feathers float on the breeze. I am told they symbolise freedom and my breathing keeps them floating.

NEUROPRO – The first image is a tropical island

I move on to the beach and find myself on the sand looking towards the statues planted to my right. But a white fog appears in front of me, which ebbs and flows as I try to focus on the faces of the statues.

That sea mist keeps returning to block my view because I’m losing my focus on the moment, and hence am not sufficiently relaxed.

Focusing on the moment takes some effort. I’m hindered by a journalist’s instinct to note events. Dr El-Imad tells me that our minds wander for 50% of our waking hours so excluding other thoughts is a challenge we all face.

I capture the statues in sharp definition from time to time and emerge shocked that I was in the machine for five minutes. It felt like two or three as I narrowed my thoughts on the scene in front of me.

NEUROPRO – If the user’s mind wanders a mist obscures the statues

Beside this set-up a laptop screen reflects the signals the EEG captured leaking out of my brain. It shows areas of my consciousness lighting up. This data is saved for analytic purposes. Anonymised and encrypted it can be sent on to neuroscience researchers.

I’m given a score that defines my level of concentration and provides a target for improvement. Dr El-Imad thinks my neurons have behaved pretty well for a first outing, rating 30% concentration.

The sense of escape that comes with sliding into a floating island vision did dilute the anxiety spurred by my tube journey. And the hovering Easter Island stage set is just one virtual location option among promised relaxing spots.

Rolling out the Dream Machine to the public should not be expensive. NeuroPro uses off-the-shelf technology, with VR and EEG hardware costing no more than £1,000 in total.

Possible sites for this service include gyms and fitness centres. Alternatively, it could be installed in chill-out rooms in corporate offices, relieving employees of anxiety and allowing the company to retain good talent.

GETTY IMAGES – Alastair Campbell has lectured businesses on the significance of mindfulness

This is where Alastair Campbell, former Labour Party communications chief and mental health campaigner, thinks the technology might have a real impact.

Mr Campbell has lectured businesses on the significance of mindfulness and says the corporate world’s view of mental health has changed a lot.

„We’ve underplayed the importance of how employees use their brains. I’ve definitely seen a change. In the City banks now realise that it’s not smart if they invest a lot in someone who then burns out.“

Quelle:

Foto: NEUROPRO – Dr Jamil El-Imad (right) worked in the computer industry before moving into neuroscience

https://www.vrroom.buzz/vr-news/lifestyle/training-your-brain-relax-virtual-island

14. Oktober 2020 0 comment

Hardware Knowledge

How to make electrical muscle stimulation feel natural to virtual reality users

If there’s one familiar sound whenever a volunteer tries out an interactive device that uses electrical muscle stimulation, it is probably laughter. Even for experienced users of the technology, the sensation of a machine controlling your body feels unnatural and strange. Something about the experience disrupts people’s sense of agency—the feeling of being in control of one’s actions—which could interfere with the technology’s potential to improve learning and make virtual reality more realistic.

As Asst. Prof. Pedro Lopes explored devices through a human-computer interface lens, first in his doctoral work at the Hasso Plattner Institute in Germany and now at the University of Chicago, he grew interested in whether agency can be measured, controlled or even restored during use of such devices. In two recent pioneering papers, Lopes and collaborators have been on the trail of agency, using everything from pitching machines to fMRI brain scanners.

„We started by asking the question: Does electrical muscle stimulation always have to feel that unnatural, or is there anything we can do to make it feel more in tune with your own volition?“ Lopes said. „I think we just started to answer it, but there are infinite ways to look at this thing because it’s such a philosophical question.“

The research on agency started with a simple demo performed at a 2018 conference by Jun Nishida, now a postdoctoral researcher at the University of Chicago: Could electrical muscle stimulation help people catch a marker dropped by another person at short range?

It did, and as expected, most volunteers attributed the action to the machine, not their own reflexes. But a small minority of participants disagreed, saying that the stimulation, known as EMS, must not have been on because they caught the marker unassisted.

Researchers examine how timing haptic actuation allows users to feel more in control. Credit: HCI lab UChicago

Those outliers inspired an experiment where Lopes, Nishida and Shunichi Kasahara of Sony Computer Science Laboratories adjust the timing of when EMS triggered a user to perform a particular task, such as hitting a button in response to a stimulus, or more advanced actions such as photographing a fast-moving baseball. They found that there’s a window of time wherein it becomes very hard for users to distinguish between their own actions and those generated by EMS.

„If we do it very early on it will feel very artificial, because you do something completely superhuman: You hit a ball much faster or you grab the pen super early,“ Lopes said. „But if we would do the stimulation just a little bit before you would normally do it yourself, you might not even spot that this was the muscle stimulation and not you. You feel like you did it.“

In that study, subjects were asked verbally after the task whether they felt like they did it themselves or were helped by the EMS device. In a recent paper, Lopes went deeper, hearing directly from participants‘ brains whether they felt agency by measuring its electrical activity during a virtual reality task.

In a virtual reality environment, users were asked to touch a virtual box. Precisely when their fingers contacted the virtual box they received either visual feedback, or visuals accompanied by vibrations on their fingertips, or all of the above accompanied also by muscle stimulation. But in a certain subset of trials, that feedback was „wrong,“ occurring before the user appeared to touch the box. When this mismatch occurred, researchers saw a notable change in the brain’s electrical activity—a neural signal that the virtual reality experience was less realistic.

„Your brain has a pretty good detector for whether you feel like you’re there,“ Lopes said. „It’s kind of dynamically measuring how you feel in those virtual worlds, even though you don’t have to report it.“

That project dovetailed with a concurrent collaboration between Lopes and neuroscientists at University College London & FU Berlin, who wanted to use EMS and fMRI to search for brain areas that distinguish between self-generated and externally-generated motion. The question reflects the selective nature of touch, where some sensations, such as from your fingers while searching for a coin in your pocket, can be extremely sensitive while others, such as the motion of your arms to and from the pocket, are virtually imperceptible.

Lopes helped design a system where participants in an fMRI scanner were asked to flex their middle finger with or without the help of EMS, receiving a tactile stimulus that simulates touching a hard surface on half the trials. This setup allowed researchers to tease apart the motor and sensory systems of the brain, and search for the brain area that potentially tunes these responses. The data pointed to a region called the posterior insular cortex, which responded strongly to touch during self-generated motion, but not during EMS stimulation.

„It suggests there’s actually a mechanism associated to touch that is checking for agency and regulating these inputs, which could explain why we can so quickly change between one thing to another, if it’s top-down,“ Lopes said. „That little process seems to know what’s going on.“

Zeroing in on this brain area and its electrical activity could guide engineers designing the next wave of EMS devices. Monitoring a user’s brain activity might help developers know, without asking, whether a virtual reality experience is perceived as real, so they can fine-tune the software accordingly. More broadly, preserving a sense of agency in the user might make EMS more effective at teaching someone how to perform a physical action, such as improving a tennis stroke or playing an instrument.

„The next thing we’re trying to see is, using these precise timings that help you be a little bit faster but still provide you with a lot of agency, does this influence the way you would learn with such a system?“ Lopes said. „It wouldn’t dramatically improve you, because we know that if it’s too much change from your expectation, you’re not going to believe it. These smaller changes might make you actually feel like you’re improving and give you a lot of chances to get better.“

Quelle:

https://techxplore.com/news/2019-07-electrical-muscle-natural-virtual-reality.html

14. August 2019 0 comment

Knowledge Research

Valve Psychologist: Brain-computer Interfaces Are Coming & Could Be Built into VR Headsets

Valve’s resident experimental psychologist Dr. Mike Ambinder took the stage at GDC 2019 today for his talk on the state of brain-computer interfaces (BCIs) and how they’ll inform the future of game design. In his talk, Ambinder put forth a near future where VR/AR headsets are kitted with non-invasive electroencephalogram (EEG) devices that could one day provide data to game designers so they can create a new generation of smarter, more reactive games. The long-term view is definitely trending towards neuronal implants, Ambinder says, but we’re not there yet.

Ambinder explained that many of the traditionally measurable ‘bio-feedback’ such as heart-rate, galvanic skin response, eye movement, facial expression, muscle tension, and physical posture are all important to understanding what’s happening to the player at any given moment, although he argues that reaching into the core of the matter, the brain itself, is the frontier that holds the most promise for game developers.

Brain-computer interfaces are basically a communication method that translates neuronal signals into actionable input for computers. In the context of gaming, the Ambinder’s envisioned goal is to acquire physiological data from a player and use that to inform the game whether a player is happy, sad, frustrated, bored, focused, distracted, etc; the idea is to figure out why that happens in each circumstance—be it anger from experiencing a game-breaking bug, or happiness when a specific goal is achieved—and build systems to leverage this data.

And it’s not as far-fetched as you may think, Ambinder maintains:

“We can measure responses to in-game stimuli. And we’re not always getting [data] reliably, but we’re starting to figure out how. Think about what you’d want to know about your players. There’s a long list of things we can get right now with current technology, current generation analysis, and current generation experimentation,” he said.

Moment-to-moment insights into the user’s cognitive state could provide a sort of adaptive, immersive gameplay which could include variable game difficulty, on-the-fly AI responses tailored to the player’s mindset, and even replacing traditional input systems all together—all within the purview of BCI’s gaming future, Ambinder said.

In the near-term, Ambinder sees EEG as the easiest way to start collecting that sort of psychological data and organizing it into something actionable for developers to create more immersive games. In the case of EEG, electrodes are non-invasively placed on various points of the scalp, and used to measure voltage fluctuations of the brain’s neurons. Things like attention, learning, memory, and intention have been measured in some respects using EEG, and Ambinder hopes these things, which have previously been the subject of scientific research, will make their way into mainstream game design at some point. But people aren’t just going to wear a full 35-electrode location EEG helmet like the one from the OpenBCI initiative on its lonesome though; that’s where AR/VR comes in.

“If you’re going to measure brain signals, you need a way to get people to wear a helmet. If only us as game designers had a way of doing that,” Ambinder said jokingly, showing a slide of an HTC Vive headset. “One advantage of AR and VR as well is you’re getting consistent contact with the source of brain activity. So you might be able to do interesting things if you could convince them to wear a helmet with EEG sensors.”

Companies like Neurable have already begun productizing EEG devices especially built to work with VR devices, letting people “control software and devices using only their brain activity,” the company claims.

That said, EEG data isn’t a perfect solution. Ambinder compared it to sitting outside of a football stadium and trying to figure out what’s happening on the field just by listening to the intensity of the crowd’s reaction. The current generation of BCI devices are noisy, and EEG is one of the noisiest due to its job of picking up neuronal signals through the skull, scalp, and hair.

There’s still plenty to learn from EEG, Ambinder says, although deeper knowledge and a push towards more immersive, adaptive games would likely necessitate invasive brain implants. Before that happens, they’d need to pass what he calls the ‘Lasik threshold’ though—a procedure that is technically invasive but confers enough potential benefits to the user while minimizing the procedure’s overall risk.

Ambinder had nothing to announce regarding the company’s plans for potentially kickstarting BCI with something as monumental as a consumerized virtual reality-EEG headset (or brain implants for that matter), although he sees EEG as a definite early avenue for capturing the sort of large data sets needed to start building out interaction systems that could span everything from creating better tutorials by learning about the user’s individual ability to learn, to making games capable of learning how players want to play as opposed to how they should play.

Making this knowledge more granular (and accurate) is still in its early phases, something that needs both sufficient time and a large enough userbase to gather data across a wide set of people. There’s a long list of powerful uses and equally powerful pitfalls yet to come.

If you want to get into the nitty gritty, Ambinder’s full talk will be up on the GDC’s YouTube page in the coming days. We’ll toss the video down here when it comes, so check back soon.

Quelle:

Valve Psychologist: Brain-computer Interfaces Are Coming & Could Be Built into VR Headsets

27. März 2019 0 comment