Tag

Audio

What are the Benefits of Spatial Audio for Enterprises?

The CEO of ConquestVR explores the benefits of additional immersive audio hardware

ConquestVR (virtual reality) is an immersive audio hardware manufacturer that sells accessories for VR headsets, including Meta Quest, Rift-S, Pico Neo 2, Neo 3, HTC Vive Focus 3 and Varjo Aero.

Prospect buyers can pre-order the firm’s Conquest Pro Headphones for $179.99. The device outputs audio from 20 Hz up to 40,000 Hz. Additionally, the hardware contains an efficient driver design that amplifies spatial audio at a lower power input.

ConquestVR is designing its upcoming product for numerous VR experiences like gaming, training, and remote communication. According to the firm, realistic audio can improve an immersive experience by boosting a user’s sense of presence, situational awareness, and motor performance.

In late October, XR Today’s Rory Greener sat down with the CEO of ConquestVR, Raghu Bathina, to discuss the importance of spatial audio in enterprise environments.

What is Spatial Audio?

Experts like Raghu define spatial audio under various terms, including immersive audio or 3D audio. During the conversation, Bathina noted that spatial audio includes several vital components.

One of those components is specialization, which simulates a sound’s direction and distance.

Users will hear audio from the left and right in conventional stereo setups, although spatialized audio advances this by enabling users to listen to sounds from all angles, above, below, in front, and behind.

Bathina added: “So it’s like a 360. That’s why it’s also called 3D sound.”

Bathina explained that another aspect of spatial audio is environmental sound, which includes elements like reverbs, occlusion, and the impact of texture on the ground, such as a carpet.

The CEO explained,

“The kind of hall you’re in can give an echoing effect. All of these take into account the environment, and then there are even more advanced features of spatial audio, such as the Doppler shift or the Doppler effect when vehicles pass by. By something known as distance attenuation, which, very simply is how far an object is from you, it [the sound] should sound quieter or louder”

The CEO also added that spatial audio and its foundational components create a more realistic environment similar to real life.

Creating Presence with Spatial Audio

Presence is essential to a successful immersive experience, including virtual, augmented, or mixed reality (VR/AR/MR) applications. Bathina said that whether for entertainment or learning, a benefit of spatial audio is the technologies ability to simulate a sense of presence.

He said: “Because of the creation of a realistic audio experience, your brain just tells you that you’re actually there.”

ConquestVR enables its hardware adopters to leverage sophisticated spatial audio experiences using its product portfolio. The firm’s product is also compatible with a range of VR headsets allowing users to connect a ConquestVR device to various headset iterations.

During the conversation, Bathina added,

“So we [ConquestVR] said, let’s come up with something which really moves with you as your VR headset moves, is very comfortable to wear, that doesn’t create a lot of sweat, and pressure on your ears. And that’s why we created the Conquest Pro headphones”

What are the Benefits of Spatial Audio for Enterprise Customers?

While spatial audio is increasing in interest within VR gaming circles, the technology is just as crucial for enterprise-grade immersive applications like virtual training, remote guidance, and meeting solutions.

ConquestVR hardware can enhance immersive workplace procedures by stopping common problems of VR headsets like sound leakage.

Bathina added,

“So, a couple of other elements are important for enterprise. One of the things is when you’re in VR [applications], given the current state of VR headsets and the way they’re designed, there is a bit of sound leakage.

The CEO explained that because of the current state of headset design,

“With our headphones, you have almost no sound leakage and almost no sound coming in from outside, so there’s very good passive noise cancellation, and that actually is very useful in an enterprise training environment where you [a trainee] may be doing the training in a more open area”

Bathina noted that overcoming issues like sound leakage was essential from a training perspective. The CEO also explained that

If you’re looking for more information on this topic, visit ConquestVR.

Quelle:

What are the Benefits of Spatial Audio for Enterprises?

14. November 2022 0 comment

Examples

Top 5 Reasons to Invest in Spatial Audio in 2022

The benefits of immersive audio experiences

Demand for immersive interactions is on the rise.

Everything from the increase of remote and hybrid work, to growing interest in the “Metaverse” has prompted rapid adoption of powerful new experiences in “Extended Reality”. However, the more companies explore the XR environment, the more they discover new demands for tools which help to embed users in digital experiences.

A truly engaging virtual, augmented, or mixed reality interaction relies on more than just phenomenal video. Sound is also a significant component in how we understand a space, or interact with content. This is why “spatial audio” is now gaining so much attention.

According to some studies, the market for spatial audio will reach around $12.97 billion by 2026, and many leading companies like Apple, Microsoft, and Sony are already getting involved. Here are some of the major reasons why businesses with an interest in XR should consider investing in spatial audio.

1. Mimic Genuine Real-World Experiences

In an age of hybrid work and remote interactions, companies and developers are increasingly working on making digital experiences feel more like real-world conversations. While video and audio conferencing tools and similar solutions can replicate many of the components of a real-world interaction, they lack a significant level of immersion and presence.

Spatial audio replicates how sound is processed in real-life, to make interactions feel more authentic. After all, the sounds people hear everyday are complex. We listen to audio in three dimensions, noticing the position, and direction of the sounds we hear. In a physical meeting room for instance, we can sense where a person’s voice is coming from when they speak to us.

With spatial audio, it’s possible to make users feel like they’re sitting in a real space through immersive technology. In a virtual meeting room for instance, it would be possible to determine who is speaking based not just on their voice, but where the audio is coming from, helping to direct user attention. This can allow for more meaningful interactions.

2. Reduced Cognitive Load

Aside from making interactions and XR experiences more immersive, research also indicates spatial audio could assist in improving intelligibility and reducing the cognitive load required when following a conversation. Without spatial information, listeners need to focus on other audio characteristics to determine who is speaking in a conversation.

While these cues are often quite effective, helping us to follow along relatively easily in a conference call or virtual event, they do require an extra level of cognitive processing. Spatial cues, alternatively, can be distinguished regardless of a person’s attention span.

Digital meeting spaces like Hubbub and Skittish are becoming more popular for virtual events, webinars, and experiences where companies want to create simple and engaging interactions. With spatial audio, interactions feel more natural and human, as well as being easier to follow. This could be particularly important in an age where digital interactions are becoming more essential due to virtual events and hybrid work.

3. Enhanced Training Experiences

Research and development into spatial audio opportunities is paving the way for all kinds of unique interactions. Today’s companies can use immersive technology for everything from meetings and webinars to virtual events. However, the right technology could also be a powerful solution for training and educational requirements.

In many training scenarios, it’s important to help users learn how to respond to situations in an authentic environment. For instance, in a military environment where companies might use XR to train team members how to respond to threats, knowing how to listen for signs of trouble and detect where these audio cues are coming from is crucial.

Spatial audio also helps to position users in a more significant three-dimensional space when they’re training. It reminds them to pay attention to what’s going on all around them, rather than just focusing on what’s straight ahead in terms of virtual reality or AR visuals. In training initiatives, spatial audio can ensure employees develop instinctual skills.

4. Enhanced Wellbeing

The concept of “wellbeing” is becoming increasingly important for business leaders and customer service experts alike. Both employees and consumers now expect brands to show a significant level of empathy, and a commitment to excellent experiences. Spatial audio in the XR landscape can help with this. According to studies into spatial audio, this technology could make interactions more accessible.

There’s a chance the cognitive benefits of spatial audio could assist those suffering from hearing loss, and make it easier for various consumers and employees to interact. Other pieces of emerging research also suggest spatial audio could help to reduce stress in various environments.

For companies investing in XR for collaboration and similar workplace purposes, spatial audio can help to fill the missing gaps in a 2D representation of audio. The right technology can help to reduce feelings of isolation in the remote world of work, and minimize cognitive stress. This could allow businesses to attract a wider range of specialists to their team.

5. Improved User and Customer Experiences

Perhaps most importantly, spatial audio can significantly improve the experience of consumers, and users implementing XR solutions. Spatial audio makes the digital world more “real” and authentic, boosting the interactions we have on a regular basis. For instance, in an environment where a manufacturing company might want to guide a specialist around a virtual twin of a workplace, spatial audio can help to make the individual feel like they’re really in the environment.

Increasingly, as audio technology evolves, we’re also becoming more adept as human beings at recognising the presence of spatial audio cues. Customers will be able to tell the difference between the psychological experience of a VR environment enhanced with spatial audio, and a lower-level VR interaction. This could make spatial audio into a powerful differentiator for brands.

As the metaverse continues to pave the way for a wider number of extended reality interactions, spatial audio could quickly become a crucial tool for enabling enhanced experiences at every stage of the customer and employee journey.

Quelle:

Top 5 Reasons to Invest in Spatial Audio in 2022

8. November 2022 0 comment

Knowledge Research

Enhancing Immersive Experiences With Spatial Audio

Why spatial audio is essential for hyper-realistic immersive experiences.

With a greater demand for hyper-realistic immersive experiences, there has been a heightened increase in interest in spatial audio. After all, it is nearly impossible to recreate real environments into digital formats without satisfying one of the primary senses—the sense of hearing. The sounds we hear are crucial to the multisensory environment needed for compelling immersive worlds.

Let’s take a quick peek at spatial audio, its accelerating adoption, and how it enhances virtual reality experiences.

What Is Spatial Audio?

Basically, spatial audio is a suite of audio technologies that deliver multidimensional and immersive sound. Compared to stereo and surround sound, which are limited to the lateral plane, it adds a vertical plane. This creates a three-dimensional sound field that envelops the listener. The result is an auditory experience that mimics the way we hear sound in real life.

When listening to music, spatial sound makes you feel as if you were listening to a live performance. In movies, it produces ambient cues that draw you deeper into a scene. It creates sounds with precise realism, elevating your listening experience to a whole new level.

How Spatial Audio Enhances Immersive Experiences

Touted to be a keystone for the metaverse, spatial audio is a core element for immersive experiences in VR, AR, and XR. Just as multidimensional images immerse us into a virtual environment, spatial sound effects also immerse us into a memorable sound experience that heightens our other senses.

A fully immersive experience requires a highly precise mix of audio cues. The visual elements we see in the virtual environment must match the detailed spatial audio mix to make the experience realistic.

Placed in specific locations both on the lateral and vertical planes are high-fidelity speakers to create a multidimensional sound. The optimized placements enable sound cues to come from any angle around the user. This creates an intuitive relationship between the digital environment and the physical space the user is in.

In highly immersive environments, such as in virtual reality applications, the use of spatial sound effects significantly impacts user experience. The experience looks more real, sounds more real, and feels more real.

Accelerating Adoption of Spatial Sound Across Industries

Beyond immersive experiences, we see an accelerated adoption of spatial sound. It is fast becoming an industry standard for platforms and brands.

Popular streaming platforms Netflix, Amazon, and Apple already have spatial and immersive categories among their services. Mercedes-Benz has immersive playback systems on some of its models. Even popular artists are releasing albums in immersive formats. Dolby Atmos has home theater systems that enable consumers to enjoy spatial sound in their own homes.

Spatial sound is best experienced using smart speakers, sound bars, overhead speakers, and other such equipment. However, you can still enjoy it on regular headphones. For instance, Sony 360 Reality Audio is compatible with most headphones and earbuds, making it ideal for music streaming with spatial sound on the go. Apple Spatial Audio, which works with AirPods Pro and AirPods Max, utilizes built-in gyroscopes and accelerometers to make the sounds you hear change as you move your head.

The increasing accessibility and affordability of spatial sound solutions are further accelerating its adoption across industries and in the consumer market. In the field of virtual reality and the metaverse, we’re bound to see spatial sound becoming the norm.

Spatial Audio: A Key Element for Truly Immersive VR Experiences

True immersion in digital environments isn’t only about what we see. It is a multi-sensory experience that also involves what we hear and feel. Thus, spatial audio becomes a key element for truly immersive VR and metaverse experiences.

Quelle:

Enhancing Immersive Experiences With Spatial Audio

13. September 2022 0 comment

Knowledge Research

Fully Immersive Spatial Audio and Why It Matters In VR

Fully Immersive Spatial Audio has the potential to change the face of the VR industry as we know it.

It’s not a new technology by any means, and it’s been around for at least 100 years prior to this moment – but it’s never quite been applied to the development of Virtual Reality quite like now, and we’re seeing great leaps and bounds in technology and the way people are immersed in their virtual worlds.

Virtual Reality becomes pretty useless to the user when it isn’t an immersive experience, and the burden falls on the developers of VR hardware and software to ensure that the experience is as close to real-life as possible. It has to make you feel like you’re right in the middle of what you’re looking at – and it’s not enough to be looking at a scene through VR glasses when you want the full experience.

True immersion for Virtual Reality involves being able to engage all of the senses at once – sight, smell, taste, touch and sound; though we know that taste and smell aren’t nearly as necessary for VR to work as the rest. Could we go as far as to say that the senses required for VR immersion is sight, touch and sound?

How do you judge whether or not something is real? In the first Matrix movie, Morpheus asks Neo if what he can smell, taste or touch is real – and then goes on to show him that the construct of “real” he had known was just about fooling these human senses with computerization.

While we haven’t blacked out the sun yet like in the heavily Dystopian tone of the Matrix movies, the movie got the idea of Virtual Reality pretty spot on. If you’re going to fool anyone into thinking it’s real, fool the senses.

To believe that you’re truly immersed in your virtual reality experience, sound and how you experience it is one of the most important parts of the whole VR thrill.

Reacting to Sound

Sound activates a primal reaction within the brain, and sound is often the very first way we react to something; for example, while you’re playing a first person shooter, you’ll be paying the most attention to the sound to find the position of the opposing player – on the other hand, while immersed in a VR movie, your mind will pay the most attention to the sound to associate both the sound and the visual aspects with each other.

To put it in its most simple terms, the brain asks itself three questions:

What am I seeing?

What am I hearing?

Do these two things match up?

The first two questions should be answered by the VR experience already – and if one of the first two questions aren’t immersive enough to the user then the brain’s answer to the third (more important) question that tells your brain how immersed it is (or not) will be a simple no.

We already have the visual aspects of virtual reality developed to a much more advanced degree than some years ago: Virtual Reality goggles are becoming cheaper and easier to access; projects like Google Cardboard have even made experimentation with VR easier.

But when it comes to developing the sound aspects that fit with the visuals, fully immersive spatial audio could be the answer to finally providing the perfect VR experience.

Why Regular Audio Isn’t Enough

If you want a practical example of why regular audio isn’t enough for real VR immersion, then you’re going to need to do a little experiment. You can use even the cheapest headphones for this.

There are many songs that illustrate the concept of stereo practically so that you can hear why it matters: Many classics by the Grateful Dead or Led Zeppelin use effects that seem to “bounce” between one headphone to another. This effect is the stereo effect – and it sounds pretty damn cool, especially through the right quality headphones.

But that’s not all it is: It’s also something very important to your brain.

Audio tells you where things are. If you were to close your eyes and listen, your brain relies on the position of sound to form a clearer picture – and the same applies to VR, when your visual input is being replaced with a picture seen on a VR screen.

In order for your brain to be truly immersed in the VR experience, your brain needs to be able to relate to 3-dimensional sound – sound that tells you more about the environment you’re in, which is (in this case) a virtual one.

If the audio is off, your brain will notice that something is “wrong” and it just won’t sound right.

When It “Doesn’t Sound Right”

When things around you don’t sound quite “right” to your brain like we’ve just explained above, your brain automatically assumes that there’s something wrong with the entire environment – and the whole VR setup starts to disappear for your brain, much like the glitch in the Matrix when the cat appears twice.

“It means they’ve changed something.” Your brain knows something is wrong when the audio doesn’t match, when it doesn’t truly immerse you in what you’re seeing – but when your brain trusts the two inputs fully, then it tends to trust the entire concept of immersion more.

You can fool your brain into experiencing a fuller virtual reality world – but only if you use sound immersion right.

What the Hell is Fully Immersive Spatial Audio?

Fully Immersive Spatial Audio isn’t a “new” technology by any means; it’s something sound engineers have known about for decades – but we’re seeing more and more reason to use this technology within VR because it offers a far more “full-bodied” sound experience that gets us a little closer to experience real virtual reality.

Fully immersive spatial audio makes use of several different techniques in order to round off the sound experience, turning it into a truly 3D sound experience that “positions” different sounds in the room – or virtual environment – so that your brain forms a much fuller image.

There are many techniques used in the development of fully immersive spatial audio, including the use of heightened speakers that add an element never-before-seen in speaker systems: Height.

Think about your average “surround sound” system – it surrounded the viewer as much as it possibly could, but one of the elements that was very rarely accurately added was height: Sure, you could tell that a sound was coming from one side or another, but you couldn’t tell the finer details like how close to the floor this sound might have been.

Are you starting to see the difference between simple surround sound and fully immersive spatial audio?

Fully immersive spatial audio offers three dimensions, and space to sound where you previously wouldn’t have had such a deeply immersive experience: It offers height, depth, reality: The real elements that you’re looking for when we’re talking about VR.

Immersive Sound and Training Developers

The applications of immersive sound for achieving real reality within VR are immense – and it also opens up a whole new world of opportunity for both sound engineers and developers, who might have to explore the field of immersive sound more in the form of workshops.

There are plenty of workshops (and them some) that explain the basics of immersive sound, how it works and how they can apply practical techniques to their sound recording (and playback) to ensure for a better sound experience.

Scaling Through Sound

Sound tells you everything you need to know about your immediate environment, and spatial immersive sound attempts to fill in all of the gaps that your brain needs to know about the essential elements.

Where am I in relation to the sound?
How far is the sound from my immediate position?
How high or low is the sound?
What’s the distance of the sound?

If your brain doesn’t receive a 3D answer about these questions – one that positions the brain, the sound and the environment all at once – then the experience just can’t be called immersive enough to be called real Virtual Reality.

The importance of fully immersive spatial audio can’t be underestimated for anyone who needs to have the fully immersive experience. If you want to fool the brain completely, it turns out that you have to fool the right senses at the same time.

Quelle:

https://www-bbntimes-com.cdn.ampproject.org/c/s/www.bbntimes.com/science/fully-immersive-spatial-audio-and-why-it-matters-in-vr/amp

19. Oktober 2021 0 comment

Examples

3D-Audio in Online-Meetings

Der virtuelle Meetingraum „Breakroom“ hat 3D-Audio von High Fidelity in seine virtuelle 3D-Welt für soziale und geschäftliche Veranstaltungen integriert.

Hinter der Entwicklung der 3D-Meeting-App Breakroom steckt Sine Wave Entertainment. Bei seiner Invention hat Sine Wave einen 3D-Voice-Chat von High Fidelity in seine Lösung integriert. Durch den 3D-Voice-Chat bekomme man den Eindruck, dass sich der Gesprächsteilnehmer im selben Raum befinde. „Der wirkliche Vorteil von 3D-Audio in einer virtuellen Welt wie dieser ist, dass man viele Unterhaltungen gleichzeitig führen kann“, sagte Adam Frisby, Chief Product Officer und Mitbegründer von Sine Wave. Es würden sogar unterschiedliche Entfernungen simuliert. Benutzer können innerhalb der Lösung auf verschiedene Avatare und abwechslungsreiche Hintergründe zurückgreifen. Breakroom steht als iOS- und Android-Version zur Verfügung und soll bald für Konsolen und die Oculus Quest 2 verfügbar sein.

Quelle:

Foto: Sine Wave Entertainment

Breakroom teams up with High Fidelity to bring 3D audio to online meetings

13. Juli 2021 0 comment

Knowledge Research

Zylia’s 3D Audio Technology Could Be The Start Of Virtual Concert Halls

A collaboration with the Poznań Philharmonic Orchestra signals a new era in live-recorded audio for virtual reality creators.

The immersive audio experts over at Zylia are all about pushing the boundaries of 3D audio recording, whether it be for academic research, art, games, music, sports, VR, and everything in between. Now the company is upping the ante even further with 6 Degrees of Freedom Navigable Audio capable of generating 3D sound for “virtual concert halls,” an immersive auditory experience unlike anything available at the moment.

ZYLIA 6 Degrees of Freedom Navigable Audio, as it’s referred to by the company, allows operators to record musical performances, Q&A’s, panels, and other live audio by capturing all audio within the spherical sound field of each and every microphone and recreated in VR. What does this mean exactly? Put simply, Zylia’s 6DoF Audio solution allows you to create your own audio path by physically moving throughout your real-world environment with a VR headset or a mouse and keyboard if operating on a computer. Each time you change positions in relation to the sound sources, the audio will change in volume and direction, just like it would in real-life. Music too loud? Take a few steps back and enjoy the show from the nose-bleeds. Looking for a more intense experience? Join your favorite artists on-stage and jam shoulder-to-shoulder.

“The mission of our company is to help artists and to bring artists and their audience together. That’s why we developed this navigable audio system in 6 Degrees of Freedom – to connect them in the virtual world,” said Tomasz Żernicki, CEO of Zylia. “When we look into the future, what we see is a completely different world of interaction with creative art. We see streaming platforms enabling people to participate in concerts of any artists they want.”

“Platforms that they can enter from their homes and join the experience with their friends. We see hyper-realistic games, both visually and audibly, allowing players to fully immerse themselves in the plot,” he adds. “Finally, we see virtual reality creations, available for everybody from anywhere in the world and limited only by the imagination of their designers. ZYLIA Navigable Audio is a huge step in that direction, and we are very proud of the impact that we are having on the future of 3D audio technology.”

Earlier this year, Zylia showcased the true power of its 6DoF audio technology in a collaboration with the Poznań Philharmonic Orchestra in Poland in which the company used 30 ZYLIA ZM-1S spherical, ambisonic microphones with synchronization to capture a 360-degree soundscape of a live performance. This third-order Ambisonics microphone features a 19-capsule microphone array with high-end 24-bit recording resolution. That means crisper, cleaner high-fidelity audio recordings over longer distances.

“To record the Poznan Philharmonic Orchestra performance we placed in the concert hall 30 third-order Ambisonics microphones ZYLIA ZM-1S with synchronisation,” explains Żernicki. “We put them on stands and placed them on the stage among musicians. The microphones have been connected with computers through simple USB HUBs. We used 2 laptops for recording and one PC Linux workstation as a backup. These 30 microphones gave us 600 audio channels – 20 for each ZM-1S. As a final result, we gathered 3 hours of recordings and 700 GB of audio data.”

The team then used Zylia software to mirror the physical location of each of the 30 mics within the virtual space, allowing listeners to explore this space to experience the music from multiple angles, just as they would at a live venue. You can check the full performance in the video provided above. You can also view a 360-degree demo in VR using a PC VR headset, and check out an exclusive behind-the-scenes making-of here.

Apart from ZYLIA ZM-1S, the Philharmonic project was brought to life using a set of Zylia software, including the ZYLIA 6 Degrees of Freedom Recorder, software which allows for simultaneous audio recording on ZYLIA ZM-1S microphones; the ZYLIA 6DoF Synchronizer, software used to upload audio recordings and synchronize them with one another; the ZYLIA Ambisonics Converted, software which converts ZM-1S multichannel recordings into Higher Order Ambisonics; and the ZYLIA 6DoF HOA Renderer, a plug-in that allows for the processing and rendering of ZYLIA Navigable 3D Audio content.

“The border between the musician and audience was existing for ages,” said Łukasz Borowicz, the Principal Guest Conductor of Poznań Philharmonic Orchestra. “Once you pass this border you are entering a real musical world. This kind of experience is an experience I cannot describe. You have to feel it. In the last months, we faced a problem of making concerts without a public. We’ve been broadcasting the concerts but it produced a lot of problems on how to show the acoustic of the hall.”

“This is something very important in the ZYLIA system – it makes it possible to listen to the orchestra not only from one fixed place. You can go to the balcony or the last row and acoustic changes. I think people are very open to new technologies and notice that the experience of the sound which comes to you directly is one of the best experiences you can have. Technology seeks the possibilities to reproduce this real feeling and I think this technology which was used for our recording shows that ability.”

In addition to the above-mentioned use-cases, ZYLIA’s 6 Degrees of Freedom Navigable Audio solution can also be used to create immersive multi-point 360-degree audio-video experiences. When combined with 360-degree cameras, for example, ZYLIA mics can be used to transport listeners to various parts of a live stage.

Each time they are being switched to a new listening and vantage point they’ll experience the audio from a different perspective, just as they would in real-life. Additionally, they can turn around in each spot to experience adequate immersive effects. This same technology can also be applied to livestreams; as the live operator determines the walk path, a live audience enjoys the 3D binaural audio remotely from their computer or VR headset.

The potential use-cases for such a technology are virtually limitless. Not only could Zylia’s 3D audio technology offer casual listeners a more immersive and engaging way of enjoying their favorite music, but it could also provide musicians and other performers with a more impactful way of connecting with live audiences. With the COVID-19 outbreak forcing the cancellation of nearly all live in-person performances, entertainers have begun turning to popular live streaming solutions to connect with their fans remotely.

VR technology has proven especially useful in this regard, allowing users to immerse themselves in virtual worlds and connect with their favorite artists on a more personal level. ZYLIA 6 Degrees of Freedom Navigable Audio could enhance these experiences even further, offering a next-generation 6DoF audio experience just as if not more engaging than the VR experience itself.

For more information on Zylia’s game-changing 6DoF audio technology, visit zylia.co.

Quelle:

Zylia’s 3D Audio Technology Could Be The Start Of Virtual Concert Halls

Image Credit: Zylia

21. Dezember 2020 0 comment

Knowledge Research

Hörwahrnehmung besser verstehen durch virtuelle Realität

Im neuen Schwerpunktprogramm AUDICTIVE („Auditive Kognition in interaktiven virtuellen Umgebungen“) der Deutschen Forschungsgemeinschaft (DFG) sind Oldenburger Experten an drei Vorhaben beteiligt. Die Forscher wollen virtuelle Realität einsetzen, um kompexe auditive Prozesse besser zu verstehen.

In Klassenzimmern, Großraumbüros oder in der Nähe vielbefahrener Straßen fällt vielen Menschen das Hören schwer. Um zu verstehen, wie die akustische Wahrnehmung in solch komplexen Umgebungen funktioniert, setzen Oldenburger Experten auf virtuelle Realität (VR). Gemeinsam mit Partnern leiten sie drei Projekte im neuen Schwerpunktprogramm AUDICTIVE („Auditive Kognition in interaktiven virtuellen Umgebungen“) der Deutschen Forschungsgemeinschaft (DFG). Die Akustiker Dr. Stephan Ewert, Prof. Dr. Volker Hohmann, Prof. Dr. Steven van de Par und der Neuropsychologe Prof. Dr. Stefan Debener erhalten für ihre Projekte in den nächsten drei Jahren insgesamt rund 830.000 Euro. Ziel von AUDICTIVE ist es zu klären, wie sich komplizierte auditive Prozesse wie Wahrnehmung, Aufmerksamkeit und Erinnerung mit neuen, interaktiven VR-Methoden bestmöglich untersuchen lassen.

Van de Par, der als Mitglied des Programmausschusses die inhaltliche Ausrichtung des Schwerpunktprogramms mitgestaltete, leitet gemeinsam mit Prof. Dr. Matthias Blau von der Jade Hochschule in Oldenburg und Prof. Dr. Andreas Mühlberger von der Universität Regensburg eines der Projekte. Mithilfe von Experimenten will das Team herausfinden, ob sich die Akustik von Räumen in einer virtuellen Umgebung realistisch nachbilden lässt und wie der akustische Eindruck durch die visuelle Wahrnehmung beeinflusst wird. Darüber hinaus beschäftigen sich die Forscherinnen und Forscher mit der Frage, welchen Einfluss die Audio-Wiedergabe in virtuellen Welten auf soziale Ängste hat – ob sich etwa die Furcht davor, einen Vortrag vor vielen Menschen zu halten, in einer virtuellen Umgebung reproduzieren und somit einfacher untersuchen lässt.

Ewert leitet gemeinsam mit van de Par und Dr. Virginia Flanagin vom Universitätskrankenhaus München ein Projekt mit dem Ziel herauszufinden, wie es Probanden gelingt, Entfernungen und Bewegungen über das Gehör einzuschätzen. Dies ist besonders in Situationen mit eingeschränkter Sicht oder bei Objekten außerhalb des Sichtfelds wichtig, wenn das Gehör als Warnsystem agiert. Das Team versetzt Versuchspersonen dafür mit Hilfe einer VR-Brille in verschiedene virtuelle Umgebungen. Dabei werden die akustischen Reize entweder über Kopfhörer oder über Lautsprecher wiedergegeben. In den Hör-Experimenten variieren die Forscherinnen und Forscher die Art der akustischen Quellen, ihre Entfernung und Intensität. In beiden Projekten wird das selbstentwickelte akustische Simulationssystem RAZR eingesetzt, das realitätsnahe virtuelle akustische Umgebungen erzeugt.

Das dritte Projekt mit Oldenburger Beteiligung wird von Debener und Hohmann geleitet. Die Forscher wollen ebenfalls experimentelle Studien in virtuellen Umgebungen durchführen, um zu verstehen, wie es gesunden Menschen gelingt, aus einer Vielzahl von Schallquellen die Stimme des aktuellen Gesprächspartners herauszufiltern. Diese Fähigkeit geht über das reine Hören oder Sprachverstehen hinaus, sondern erfordert, die Aufmerksamkeit entsprechend steuern zu können. Hohmann, der an der Universität auch einen Sonderforschungsbereich zur Hörakustik leitet, hat mit seiner Arbeitsgruppe ein leistungsfähiges, akustisches Stimulationssystem namens TASCAR aufgebaut. Diese Plattform ermöglicht es, virtuellen Charakteren nicht nur eine Stimme, sondern auch realistische Lippenbewegungen zu geben und alltägliche Gesprächssituationen abzubilden. Um herauszufinden, wie die Versuchspersonen ihre Aufmerksamkeit steuern, will das Team deren hirnelektrische Aktivität mit einem mobilen EEG-Gerät messen, das von Debeners Arbeitsgruppe entwickelt wurde. Ziel der Kooperation ist es, die mobile EEG-Technologie in der VR zu etablieren und beispielsweise zu untersuchen, welche visuellen Reize dazu beitragen, dass sich die Aufmerksamkeit auf die gewünschte Schallquelle richtet.

AUDICTIVE ist eines von 14 Schwerpunktprogrammen, die die DFG in diesem Jahr eingerichtet hat. Es verknüpft die Disziplinen Akustik, Kognitionspsychologie und Informatik.

http://uol.de/neuropsychologie

http://uol.de/mediphysik-akustik/

http://www.spp2236-audictive.de

Quelle:

Foto: Wie gelingt es gesunden Menschen, aus einer Vielzahl von Schallquellen die Stimme des aktuellen Gesprächspartners herauszufiltern? Das untersuchen Oldenburger Forscher mit EEG-Messungen und virtueller Realität., , Universität Oldenburg/Giso Grimm

https://nachrichten.idw-online.de/2020/11/13/hoerwahrnehmung-besser-verstehen-durch-virtuelle-realitaet/

18. November 2020 0 comment

Hardware Knowledge

Sony’s headphone app will soon analyze your ears for 360 audio

More immersive sound is nearly ready for your phone.

Back at CES, Sony unveiled 360 Reality Audio, a new standard/format/ecosystem for immersive sound on headphones and speakers. The headphone demo at CES was very technical and quite sensitive as calibrating sound profiles to your ears required a thin wire underneath some studio-grade cans. The company was clear in January that the goal would be for anyone to do the calibration with their phone, by taking a picture of their ears. At IFA, Sony is showing off how that feature will work inside its current Headphones app.

The Sony Headphones app is already the place you adjust noise cancellation, tweak the EQ and make other changes to its line of audio gear. It was nice to see the company will most likely put this 360 Reality Audio calibration tool inside that app. I have enough things one my phone, and I’m glad I won’t have to add another app for the sole purpose this one-time setup.

Once you select the 360 Reality Audio Setup, the Headphones app will walk you through the process. This includes asking you to make sure your ears aren’t covered by hair, remove earrings and, of course, remove your headphones. Next, you have to take the photos. The app will recognize your face to make sure you’re holding your phone the appropriate distance away. Then you simply turn your head to the right to capture your left ear before repeating the process for the right. The app will automatically detect when you’re in position and capture the image. If it doesn’t, there’s a manual option so you can snap a photo yourself. The whole thing is a bit like taking a selfie, but it can be difficult to get the distance and angle right, even if the phone is mounted stationary — like it was on the show floor.

When all the photos are done, the app analyzes the shape of your ears to create a sound profile. That process can take up to 30 seconds, but the few times I tried it, the software was much faster. Once complete, the Headphones app will send the results to a music app to „optimize your 360 Reality Audio experience.“ Since the whole thing is still new, so right now that means booting you out to Sony’s own media player where there’s a selection of curated 360 Reality Audio tracks. For the demo, you had the option to switch back and forth between stereo and this immersive format. But it was only for one song and not for any of the material from the likes of LSD or Khalid.

There is a massive difference between 360 Reality Audio and normal stereo. It’s like the difference between stereo and Dolby Atmos in your living room, or digital and analog instruments. It’s much more immersive, making it seem like you’re inside the music, rather than just having it blasted into your eardrums. Sony was doing this demo with its stellar WH-1000XM3 headphones, a $350 set that’s the best noise-canceling option you can buy right now. Even though those headphones are great, they’re even better with 360 Reality Audio.

The one element of all this that I’m unsure of is just how much the photo/calibration process tailors things to your ears. I tried the immersive tracks and stereo sample with the previous attendee’s settings before going through the process myself. Obviously, the whole thing takes a couple of minutes, and I wasn’t able to flip back and forth. All in all, I couldn’t really tell a difference between the two profiles. Who knows, we could’ve had similar ears. Sony reps didn’t offer any info on what’s going under the hood during the analysis, which isn’t really surprising — just that the app is creating an audio profile specific to your ear shape.

While I’ll have to wait until the feature is ready for mass consumption to really put it to the test, having 360 Reality Audio content to use it with is the next hurdle. At CES, Sony said it working major labels and distribution services, and any existing music recorded in a multi-track format could be easily converted. Still, it’s unclear if you’ll have to pay extra for it, if this will be exclusive to artists on Sony’s stable of record labels or even when you’ll be able to access it. The good news is that whenever the time comes, the calibration part of the equation will arrive as an update to an existing app.

Quelle:

https://www.engadget.com/2019/09/06/sony-360-reality-audio-headphone-calibration-hands-on/?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAAEHEzXiSlHp02SfYyhApbg7E3eDScbd7MWLaRK_pBQ2W5MTUoDMHX0z-hzn6FI29uAQVAyoRTqjTsWkJrN9i3MjeKSLBKMWWcRqRDruqWbdPR0tLc06R9Pbk7iq4_k8_Yt6Ji5wy1YVo0JVjIHqfu2jEBakOUPRFxk_1RTlIGOPg

11. November 2019 0 comment

Knowledge Research

ALGORITHMUS REVOLUTIONIERT VR-SOUND-ERLEBNIS

Ein neuer Ansatz der Stanford University ist schneller, günstiger und liefert realistischeres Ergebnis hinsichtlich des Sound-Erlebnisses in Virtual Reality-Einsatzszenarien meldet Pressetext.com.

Forscher der Stanford University http://stanford.edu haben einen technischen Ansatz vorgestellt, der das Sound-Erlebnis in virtuellen Realitäten (VR) revolutioniert. Möglich wird das durch einen Algorthmus, der die in VR-Umgebungen notwendigen dreidimensionalen Raumklänge in nur wenigen Sekunden berechnet. Das innovative Verfahren ist aber nicht nur um ein Hundert- bis Tausendfaches schneller als bisher verfügbare Methoden, sondern auch weitaus günstiger und liefert überdies qualitativ bessere Ergebnisse, versprechen deren Erfinder.

„VR IST UNVORHERSEHBAR“

„Wenn wir uns Filme ansehen oder Videospiele spielen, können die richtigen Sound-Effekte maßgeblich dazu beitragen, ein realistischeres Erlebnis zu erschaffen“, erklärt Doug James, Professor für Computerwissenschaften an der Stanford University. Ist auf der Kinoleinwand etwa ein Pokerspieler zu sehen, der einen Silber-Dollar über den Kartentisch rollen lässt, scheint es, als würde sein Klang von einem Ohr zum anderen wandern. Dieser akustische Trick wird von Filmproduzenten erreicht, indem mehrere vorab aufgezeichnete Sounds miteinander verbunden werden.

„Das Kreieren solcher Sound-Erlebnisse in virtuellen Realitäten ist praktisch unmöglich, weil VR unvorhersehbar ist. Es ist schwer zu sagen, welchen Sound ein Objekt erzeugt und wo dieser Sound gehört wird“, schildert der Experte. Um eine halbwegs realistische 3D-Sound-Kulisse in VR-Welten zu erschaffen, konnte man bislang nur versuchen, mit einer Vielzahl von Klangmodellen zu arbeiten, die vorab aufgezeichnet werden müssen. „Für die hierfür nötigen Berechnungen braucht ein ganzer Cluster von Computern mehrere Stunden. Unsere Methode schafft das in Sekunden“, so James.

NEUER THEORETISCHER ANSATZ

Der neue Ansatz aus Stanford hat einen ganz anderen theoretischen Ausgangspunkt als bislang verfügbare Verfahren der 3D-Raumklangerzeugung, die sich auf den deutschen Physiker Hermann von Helmholtz berufen. Dieser hatte schon im 19. Jahrhundert eine wichtige Gleichung aufgestellt, mit der sich die Ausbreitung von Schall im Raum beschreiben lässt. Statt Helmholtz kommt bei James der österreichische Komponist Heinrich Klein zum Zug, der den sogenannten „Mutterakkord“ entdeckte – einen Akkord, der aus zwölf verschiedenen Tönen und Intervallen besteht. Nach demselben Prinzip arbeitet auch der neue Algorithmus: Mehrere Klangmodelle werden zu einem einzigen Sound verschmolzen, der sich zeitbasiert wiederum in individuelle Klänge zerlegen lässt und so eine schnelle Berechnung von akustischen Raum-Sounds ermöglicht.

Weitere Informationen:

Ein plastisches Beispiel kann man auf Youtube anschauen:

Quelle:

https://www.it-daily.net/shortnews/22016-algorithmus-revolutioniert-vr-sound-erlebnis

14. August 2019 0 comment

Knowledge Research

Recording And Mixing Audio For Virtual Reality

As VR applications and cinematic experiences grow in popularity, so comes a requirement to design, create and mix the actual audio for these virtual spaces.

Technically speaking, VR audio is binaural, in other words a different audio signal is fed to each ear in order to create the perception of a three-dimensional sound field. In many senses, the stereophonic sound is binaural, but the words aren’t synonymous: stereo audio can’t recreate a complete and natural three-dimensional sound field, whereas that’s what binaural audio is all about. Stereo works very well, obviously, for creating left-right positioning and – if you know the tricks – can create the impression of near-far as well.

When coupled with conventional visual media, this works just fine, because the action on a two-dimensional screen maps naturally to a stereo sound field. Surround sound creates a more immersive aural experience than stereo, but just like stereo it is locked to a fixed, two-dimensional point of view, and just like stereo, it has no way of capturing or simulating the up-down position of a sound source.

What’s needed for cinematic VR, then, is a method of capturing panoramic three-dimensional sound, and a way of mixing and reproducing that sound independently of the listening position and speaker configuration – surprisingly, the technology for doing this, Ambisonics, has been around since the 1970s.

Ambisonics is a full-sphere surround-sound format, and works on much the same principles as mid-side stereo. With mid-side, the mid channel carries a collective signal of the sound at the listening position, while the side channel carries left-right positional offset information; in an Ambisonics system, three side channels are employed to capture positional information in all three dimensions (‘higher order’ Ambisonics use more side channels for greater positional accuracy).

And because Ambisonics captures a full-sphere sound field, it’s possible for the apparent listening position within this field to be modified by, in effect, rotating the sphere – just what’s needed for VR! Similarly, when mixing for VR, Ambisonics panners allow you to position non-Ambisonic parts within the 3D sound field, in much the same way as a stereo panner lets you set the left-right position of a mono part within a stereo sound field.

Ambisonic pathways

There are actually quite a few Ambisonic microphones on the market now, and custom-built rigs of omni- and bi-directional mics are also common. Good examples of these include the Sennheiser Ambeo, NT-SF1 (and Soundfield app) by RØDE, Core Sound’s TetraMic and the Zoom H3-VR. Most mainstream DAWs support Ambisonics either natively (Pro Tools, Nuendo and Cubase, for example) or via plug-ins such as the Waves B360 Ambisonics Encoder. This support allows you to create a multi-channel Ambisonics mix while monitoring via stereo headphones and allows the listening direction to be modified, either by an HMD or via onscreen controls.

There are also tools for syncing your DAW with a VR video player, although this is a clumsy way to work because you can’t see both your DAW’s controls and the scene you’re working on at the same time, and when viewing the VR scene, there’s no guarantee you’ll even be facing your workstation. Solutions will come, but for now, this clumsiness is par for the course.

Once you’ve got DAW and HMD playing nicely together, you then have to face the biggest challenge of VR audio: nothing you know is relevant any more! Reverbs don’t really work in this all-encompassing context, and create a confusing and unrealistic aural environment; positioning point-source sounds, such as a voice or car horn is easy enough, but convincingly simulating their reflections and echoes within the virtual environment is not; positioning wider, diffuse sounds, like ambience and foley, and making them move realistically with the point of view, is also difficult but not impossible. For example, the Ambisonics mode of Steinberg’s MultiPanner has a number of tools for controlling the three-dimensional position, source size and field size of a sound.

However, capturing natural sound in a real space using a full-sphere mic will always produce a more convincing result than trying to simulate sound effects, ambience and Foley with samples and reverb; in the absence of an Ambisonic mic, a mid-side pair will capture a sound that works reasonably well within an Ambisonic system.

As with most things, you can find plenty of discussions on the web about VR audio, and these are certainly useful in helping you set off on the right course. But if you’re planning on moving more seriously into VR audio production then the best thing to do is experiment, fail, and then experiment some more! Every innovation in technology has been essentially founded on that principle. With VR technology accelerating at the rate it is, it makes sense to arm yourself with the technical knowledge that could potentially become the norm for audio production in years to come.

Quelle:

https://vrroom.buzz/vr-news/tech/recording-and-mixing-audio-virtual-reality

Foto: Ambisonic mixing requires a different understanding of how (and where) your audience will hear the various elements of your track.

25. Juli 2019 0 comment