Sundance describes the New Frontier portion of its annual film festival as a "creative space" for "multimedia performances, transmedia experiences," and other interactive art forms. This year that translated into a room of pure sensory overload: orange-red walls with matching plush couches, video projections, and in one corner, five stations gilded in chrome. That’s where a constant stream of visitors would sit down, plug in, and get transported into another world.
Oculus had come to Sundance.
Since its Kickstarter debut, much of the public focus on the Oculus Rift VR headset has been on its potential as a gaming device. The uses there are obvious, but talking to Nate Mitchell, Oculus’ VP of product, it’s clear the company sees the Rift having much broader applications in everything from immersive experiences to narrative storytelling. Opening eyes to those possibilities — and enticing other artists to dive in — was a big part of what it hoped to achieve at New Frontier. "To be invited to be at Sundance, and share the canvas and show people, ‘Hey, here are some cool things that artists are doing,’" says Mitchell. "We’re big believers in that."
Oddly enough, this wasn’t the first time a Palmer Luckey-designed headset took the spotlight in Park City. In 2012 an interactive experience called Hunger in Los Angeles took participants through a virtual recreation of a real-life event outside an LA food bank, using goggles that a 19-year-old Luckey had worked on. When I tried Hunger for myself last year, I was stunned at how emotionally arresting a VR experience could be.
"The director said, 'I just want it to be you and her.'"
The demos this year were a mix of the old and the new. Two stations were devoted to a title we saw at CES — the space dog-fighting game EVE: Valkyrie — but the Rift showed off a few new tricks when it came to the narrative section. Oculus Cinema is the company’s virtual movie-theater simulation, and while we’d seen a version before, this time it was paired with the Sundance short 130919: A Portrait of Marina Abramović. The film consists of one long, unflinching shot of the groundbreaking performance artist herself, and to create a more seamless experience Oculus had pulled the theater seats from its digital movie palace. "The director said, ‘I really wanted it to be just you and her,’" explains Oculus product lead Joe Chen. The result was a surreal, almost intimate experience, though the HD Rift I was shown didn’t quite have the visual clarity needed to sell me on the illusion of being inside a theater (the latest Oculus Crystal Cove prototype wasn’t in attendance).
Clouds is a documentary created by James George and Jonathan Minard, made up of interviews and set pieces stitched together in a series of Choose Your Own Adventure-style branching paths. Normally users take advantage of a Kinect to guide their journey, but in the Oculus version I could simply look towards an area of the screen I wanted to explore.
I realized I was experiencing a total lack of distraction
Clouds also showed some real ingenuity in the way it played with a common convention: on-screen titles. Rather than superimposing the name of a person over a given interview, Clouds tucked the information just below my field of vision. I could look down to read their name, along with a snippet of text explaining who they were, but I’d otherwise just watch them speak. It’s a small tweak, but one with a noticeable impact. Without text getting in between me and the speaker, I could just be present in the virtual space with them. I felt the same sense of intimacy that I noticed during the Abramović short, and then it hit me: I was experiencing a total lack of distraction. Cocooned inside the headphones and Oculus goggles, I didn’t have to worry about my phone, noisy audience members, or the abstraction of on-screen titles, and could just focus on the experience itself. Granted, I wouldn’t want the titling trick used in every movie — there’s an element of work involved when you have to hunt around for basic information — but it was a reminder of the many possibilities that will be available as we discard the confines of movie and TV screens for immersive storytelling.
Curiously, the most exciting demo of the bunch was one that relied on sound to work its magic. Last year Beck teamed with director Chris Milk to film a 10-minute cover of David Bowie’s "Sound and Vision" with an elaborate backing orchestra. The performance was turned into an interactive web experience, allowing users to switch between three different rotating camera positions, but the real trick was its binaural soundtrack: the perspective of what you heard would change depending on where you were looking. Milk ported it to the Rift, and the result is stunning.
With a finger clicker I could toggle between the three different camera positions, and was able to look around with a full 360-degree field of view at all times. When the orchestra’s strings were behind me, they sounded like they were behind me. When I looked over my shoulder to watch the vocalists, their voices swung around to match. Sound provides crucial information that helps us understand the world around us, but it’s easy to take for granted. Diving into "Sound and Vision" made it fundamentally clear how important audio is going to be to the success of VR experiences.
"They come up with things that we wouldn't have even thought possible."
"They challenge us, because we’re just making hardware and software," Chen says of the artists and developers building for the Rift. "So by showing us their vision, they come up with things that we wouldn’t have even thought possible … And that dialogue is ultimately going to not only progress the art, but also progress and challenge us on the tech."
Everyone that’s been able to strap on the Rift understands the potential. What people will actually do with the headset remains the great unanswered question. The Sundance demos offer a hint of what’s next: each contained one or two of what could be considered minor innovations, but all of them together — compounded by the lessons learned from every other Oculus project — are slowly beginning to form the collective vocabulary of VR design. Whether it’s removing theater seats or creating 360-degree sonic soundscapes, those ideas and concepts are shaping an entirely new medium with unknowable potential — right here, right now.