In the first episode of The Future of Music, I attended an Imogen Heap concert... by putting on an Oculus virtual reality headset. This concert, which takes place in a VR model of her childhood home, is a collaboration between Heap and Los Angeles-based company TheWaveVR. While Heap sings, I walk around, wave at, talk to, and make friends with other people in the room who are represented by cute animal avatars like a fox or bunny. The kicker to all this: I’m not watching Imogen Heap perform but a 3D hologram-like version of her.
Seeing your favorite artist in person is one of the best feelings in the world, but virtual reality concerts are rapidly becoming a viable alternative. Over the past several years, artists like Ariana Grande and Snoop Dogg have experimented with recorded versions of their performances that can be viewed in VR headsets. Viacom worked with Billy Corgan to create a biometric VR video for his song, “Aeronaut.” Live Nation even has an entire division for VR concerts called NextVR, and Oculus Venues recently hosted its first live VR concert with Vance Joy featuring a limited social component.
However, all of these experiences feel like they’re trying to replicate real-life concerts, not reimagine them, making it hard to justify why someone should choose to watch a concert in VR over going in real life. But, TheWaveVR thinks it’s cracked the code. Instead of replaying concert footage in VR headsets, it collaborates with artists to create custom, psychedelic VR worlds where users can interact with each other while at the same show.
Psychedelic VR worlds where users can interact with each other
This is the two-pronged approach TheWaveVR believes VR concerts need in order to make them a different category: visuals you can’t get at a real show and a true social experience. A large part of what makes concerts so fun and memorable is that you get to share them with other people and make new friends. At present, most VR concerts are little more than expanded real-life videos you watch by yourself on expensive equipment most people still don’t have access to.
I’ve known about the TheWaveVR for some time, and I first met CEO and co-founder Adam Arrigo in VR last year within an experience the company created for electronic artist TOKiMONSTA. As glowing little bunnies, Adam and I teleported around the virtual planet it had created for TOKiMONSTA’s album Lune Rouge. Together, we played with an alien-looking tree that let us remix one of her songs by picking up glowing orbs. This sense of play, interacting with music, and being together with Adam within the VR space piqued my interest, and I wanted to know more.
Here’s how it works: TheWaveVR hosts live events all the time, both with established artists and with community members who can schedule their own shows. They have a default set of graphics, but with its Wave Builder tool, anyone can create their own VR venues, parties, and flashy add-ons. The DJs performing at these live events have control over the music as well as the entire venue. It can be changed from a nightclub to outer space, for example, or moving visuals can be triggered — like neon sparkling ribbons of light shooting overhead. Meanwhile, as a user, you’re free to walk around, approach and talk to other users, and teleport from party to party. It’s essentially a dream rave from your living room.
It’s essentially a dream rave from your living room
While the main platform is open to everyone, TheWaveVR also partners with artists for bespoke VR projects. There’s the aforementioned TOKiMONSTA world and another show featuring Iranian artist Ash Koosha, who was unable to come to the US due to Trump’s travel ban. “We let him perform his show for the first time to a US audience who was exposed to this totally different type of art, both musical and visual,” Arrigo tells me. “We think it can close borders, and that’s what’s really exciting.” These artists are actually performing live within the VR space. That’s not true of Heap’s elaborate 3D rendering. But TheWaveVR hopes to run fully live “hologram” performances in the future.
TheWaveVR’s newest collaboration, which I got to see during the build process, is with Grammy Award-winning musical artist, producer, and audio engineer Imogen Heap. (You might know some of her hit songs like “Hide and Seek” and “Let Go.”) Heap’s always had an interest in tech. Over the years, she has spearheaded ventures like Mi.mu, her sensor-enabled gloves for performing music, and Mycelia, a blockchain project for musicians. Now, she’s tackling virtual reality. “The real magic came when I was interacting with Adam,” Heap tells me. “I realized that I felt so much closer and more connected, that it wasn’t this distant magical space. It was this real, human interaction.”
Logged into Heap’s concert is a breathtaking experience. It’s not as wild and otherworldly as some VR experiences I’ve had in the past. This is intimate by design. I’m in her living room — or rather, bits and pieces of her living room — while Heap’s image appears to be right in front of me, as if I could reach out and touch her. As she sings, the walls around her splinter apart, and her body begins to dissolve while she rocks back and forth. “This show, unlike our others, is all about the hologram and the audience’s connection to her,” Arrigo tells me. “So VR can afford this unprecedented level of intimacy with a performer because you can be more close up to them. You can experience the music in a new way.”
For TheWaveVR’s part, this project is particularly special because of its vibrant, three-dimensional likeness of Heap. TheWaveVR went to Heap’s home in London and recorded a terabyte of information at her 40th birthday party, including depth video, audio, data from her Mi.mu gloves, and other sources. These depth videos were captured with Scatter’s Depthkit volumetric video tool and are an integral part of creating the hologram-like effect. Depthkit records a distance for each pixel, allowing 3D information about an object to be recorded via something like a Microsoft Kinect sensor. This is then synced up with an actual video camera to overlay a texture on top, and ta-da, there’s Heap in 3D singing “Let Go.”
While she’s singing, Imogen might break into a million grains of glowing dust
All this data isn’t about making a true-to-life replica of Heap. Instead, TheWaveVR uses it to manipulate Imogen’s image and add visuals to the performance you can’t get in real life. While she’s singing, she might break into a million grains of glowing dust or dissolve into orbs of light. The home around her transforms, too: the walls crackle upward and the floor warmly glows like pools of lava.
Despite all these effects, Heap tells me she wanted to keep the experience grounded in reality. “I think you don’t need to wow people,” Heap says of how she chose to craft the experience. “I think that can happen in the VR space, where you get bombarded by all this new stuff hitting your senses all the time, and it’s actually quite exhausting, and a bit shouty. I think I’m really enjoying how to bring everything really close.”
VR has come a long way over the past few years, but even though VR experiences are more common, it’s nowhere near being mainstream. It’s expensive, can be bulky, and requires a high-end wired headset for a truly immersive experience. Many VR creators have emphasized that they’re making a long-term bet. Harmonix CEO Steve Janiak, whose studio is behind experiences like Rock Band VR and Harmonix Music VR, told The Verge last year that widespread adoption could be years down the road. “But it’s such a transformative technology, and the kinds of experiences you can deliver are so unique in VR that I find it difficult to think that it won’t catch on eventually,” he said.
Virtual concerts can’t replace the live show experience
Virtual concerts can’t replace the live show experience — the atmosphere, the feeling of your best friend’s arm over your shoulder while you shout out lyrics together — but it can provide something different. That “something different” is what TheWaveVR wants to define as VR becomes more accessible to the general public. Projecting a recorded video performance into an Oculus headset is fine, but in many ways, it’s an isolating limitation. The VR space is an open playground, so why not treat it like one? That’s the future that TheWaveVR and artists like Imogen Heap see.
“Our central mission is about parsing the power of VR technology and leveraging it to create a visual experience, an interactive experience, a social experience that you just couldn’t have at a real show,” says Arrigo. “So we think that the future of music is probably something that combines all of those elements. It’s not just about how music gets performed or distributed in a new way, but how people socialize around it [and] how it’s visualized.”