A new virtual reality system developed at the University of Birmingham could transform how musicians practice, perform and teach remotely by solving a critical technical challenge that has long plagued online music collaboration: latency.
The Joint Active Music Sessions (JAMS) platform creates responsive avatars that enable musicians to play together in perfect synchronization through VR headsets, eliminating the tiny delays that can derail a performance. According to Dr. Massimiliano (Max) Di Luca from the University of Birmingham, even minimal latency can be disruptive: “performers can start to feel the effects of latency as low as 10 milliseconds, throwing them ‘off-beat’, breaking their concentration, or distracting them from the technical aspects of playing.”
The system works by having musicians record themselves playing, which generates personalized avatars that other performers can interact with in virtual space. “The software creates a responsive avatar that plays in perfect synchrony with the music partner,” Di Luca explains. “All you need is an iPhone and a VR headset to bring musicians together for performance, practice, or teaching.”
What sets JAMS apart is its attention to the subtle visual cues essential to ensemble playing. The platform captures nuanced details like the tip of a violinist’s bow or eye contact between performers at crucial moments. These avatars respond dynamically to the VR user’s movements, creating what the researchers describe as a “unique, personalised experience” that closely mirrors in-person collaboration.
The technology emerged from the Augmented Reality Music Ensemble (ARME) project, which brought together experts from six disciplines including psychology, computer science, music, and sports science. Their combined expertise enabled the development of precise computational models that capture both the timing adjustments between performers and the physical movements that make musical collaboration feel natural.
“We’re aiming to bring the magic of playing music in person to the virtual world,” says Di Luca. “You can adapt the avatar that other people play with, or learn to play better through practice with a maestro.”
Beyond its immediate applications for music education and performance, JAMS shows promise as a social platform where musicians can connect, create, and reach broader audiences through virtual concerts. The technology also opens new commercial possibilities through its ability to create digital twins of musicians, potentially transforming how musical content is licensed and distributed in virtual environments.
Quelle:
Foto: Musicians collaborate remotely with just an iPhone and VR headset, using software that creates a synchronized avatar for performance, practice, or teaching.