Like any Bay Area resident with a pulse, Neal Mohan is a big fan of the Golden State Warriors, but as a busy Google executive, he can’t make every home game in person. "That courtside seat is limited by the laws of physics," says Mohan ruefully. Luckily, he won’t be bound by those rules for very long. This morning, in front of the biggest players in broadcast media, Mohan took the wraps of YouTube’s newest feature: live-streaming, 360-degree video. "Now anyone, with just their phone, can have that front row experience without having to be there."
Mohan’s announcement arrives just one week after Facebook unveiled its plans for expanding live-streaming and its own design for a 360-degree camera, although it can't yet support the combination of live and 360. The two biggest players in online video are racing toward the same goal: completely immersive entertainment that transports the viewer to another world. But they are taking strikingly different paths to get there.
For YouTube, Live 360 content is the gateway drug between what most people watch today and the immersive era of virtual reality that is just getting underway. "As a user, you don’t need to do anything," says Mohan. Open the YouTube app on your mobile device or launch it on the web and you’re ready to go. "There is no fancy technology to purchase or integrate." That’s a not so subtle jab at Facebook’s Oculus Rift, which offers cutting edge virtual reality, but retails for $599, not counting the $1,000 gaming PC that powers it.
360 video is now relatively cheap to make
Consumption is one side of the puzzle, creation is the other. At launch, YouTube’s Live 360 product will work with cameras that cost as little as $350, far less than the $17,000 it would cost to build the open-source hardware Facebook showed off at F8. YouTube is also releasing an API so any hardware maker can integrate Live 360 into their product. And it’s deploying 360 cameras to its YouTube spaces so creators can start playing with this new feature.
Mohan believes it will soon be possible for consumers to get in on the game. "Right now it’s specialized hardware," he explains. "But you know how these things work from a technology curve standpoint. We think we need to have a handful of flagship experiences and then as hardware tech takes off, it will spread out more broadly."
Mohan, who spent eight years working on Google’s advertising products, was recently tapped as one of CEO Sundar Pichai’s top lieutenants, promoted to senior vice president, and given the role of chief product officer at YouTube. Listening to him talk about the possibilities for Live 360, it’s clear his new role is exciting in a way programmatic ad auctions might not have been anymore. There was the experience of walking through a faraway jungle, and the one where he sat inside a fireworks display. The Verge will be premiering the first 360 live stream on YouTube this Wednesday with Dawn Richard. "Those are the kinds of magical experiences this technology is going to be able to create," he says with a beaming smile. "Without having to spend money on some fancy headset."
360 video launched on YouTube in March of 2015 and arrived on Facebook in September of that year. But it has actually been around for more than decade. "We’ve seen this before in the multimedia age," says Miles Perkins, who worked at Industrial Light & Magic and Lucasfilm before joining Jaunt VR. "The 360 Quicktimes, if you even remember that."
Dan Rayburn, an industry veteran, was working on 360 videos during the dot-com era. "The real question is, is it a gimmick? Will people feel immersed or will they be annoyed to see the back of the performer’s head? I think a lot of people do it once and think, oh cool, and then never come back." That was in the days when your only option was a desktop PC and a mouse. "The problem was that there wasn’t a great way to consume it, and it didn’t look great. Maybe the hardware today will change that, but I don’t think so. Not yet."
Lots of 2D, monoscopic 360 video is labeled as VR, but Perkins is quick to draw distinction. "For people in VR, there is a little bit of annoyance about it, like, let’s clearly define these two things. There are a lot of people who experience 360 video and assume its VR, and it so isn’t."
Many in the industry see 360 as a useful stepping stone, something that will play a critical role in helping the fledgling VR industry get off the ground, even if it’s not really what the medium was intended for. "I think 360 video is good to fill the VR content gap," wrote Alban Denoyel, co-founder & CEO of the 3D-imaging startup Sketchfab, in an email to The Verge. "But in its essence, 360 video is a format of today (flat video) stretched to fit platforms of tomorrow (VR)."
Others see shades of grey, with 360 video and true VR existing on a continuum of immersive quality. "As you start incorporating features like 3D, stereo 360, and spatialized sound, then you have something you really get lost in, and feel like you’re present in that space," said Aaron Koblin, the chief technical officer of the VR startup Vrse. "To me that’s where it gets really exciting, and where it starts to get blurred, between whether it's 360 video still, or whether it’s starting to become a VR experience."
Onstage at F8 last week, Mark Zuckerberg conflated the two as he charted the evolution of preserving and sharing memories. "When my nephews took their first steps, my sisters took photos and videos on her phone, so she could send them to us," he told the crowd. "And when my daughter Max takes her first steps, hopefully later this year, I want to capture the whole scene, with a 360 video, so I can send it to my family and my friends, and they can go into VR, and feel like they’re actually right there in the living room with us."
Streaming live in 360 is still a technical challenge
While recording in 360 is fairly accessible now, making it look good live is not so simple. "It’s something we can do for customers now, but we mostly try to talk people out of [attempting live 360 video]," says Henry Stuart, the CEO of the immersive video company Visualise. "Most consumers, their internet speed is so poor, the image ends up getting really compressed. In VR that looks especially bad."
To make Live 360 work, YouTube is announcing a slew of technical changes today. It’s upping the the resolution and frame rate that creators can send to 1440p and 60 frames per second. It’s also turning on support for DASH and VP9 ingestion, which in layman’s terms means broadcasters will need to send about half as many bits to deliver a super high-quality stream. And 360 videos viewed on Android can now have directional sound.
Right now live 360 doesn't support directional audio or stereoscopic (3D) video. But you can see how adding these two features would be the next logical step. YouTube announced 360 video in March of 2015 and support for 3D 360 eight months later. It took another six months to get support for Live 360 now, and as the technology becomes cheaper and more powerful it will almost certainly move to support Live 360 video with 3D depth and directional sound — the kind of high-end broadcast that NextVR is bringing to live basketball and boxing for Fox Sports. "You’re spot on," said Kurt Wilms, a senior product manager at YouTube VR and Live, when I asked about that roadmap. "We’re going to continue to push the boundaries and explore."
That approach — launch first, improve later — mirrors the strategy Google has put in place for VR hardware as well. Cardboard isn’t in the same league as the Oculus or Vive, but it has also shipped over five million units, several orders of magnitude more than its high-end competitors. NextVR has a more immersive live 360 stream, but it charges a pretty penny for that service, and carries its gear around in a large broadcast truck. YouTube will start out working with cameras like the $350 Ricoh Theta, the $500 ALLie, and the $1,800 Orah 4i.
"This will work with the kind of high-end cameras that folks like Next VR, but the cool thing is, this will also work with low-end stuff like a Theta," says Wilms. "You can live stream 360 of your kids concert, or set up a camera next to the sideline of a soccer game and stream it 180."
Support for Live 360 is also leading up to Google’s I/O conference next month, when it is widely expected to unveil its plans for the next generation of VR on Android. We don’t know for sure, but it would make sense for this device to sit somewhere between Samsung’s Gear VR and higher-end devices like the Oculus. With the spatial awareness technology of Project Tango, Google has already shown us that it can do really interesting VR with just a tablet strapped to your face, requiring none of the wires, motion sensors, or external processors you need for a Rift or Vive.
In a lengthy interview published this week, Clay Bavor, the head of Google’s VR efforts, reaffirmed that it wants ordinary smartphones to be at the center of its VR experience. "Sure, we could go do that," Bavor said, of an expensive and immersive headset complete with wires and a sensor array. "But it doesn’t lead directly to bringing this technology — and the best seat in the house, anywhere — to the world." Lenovo announced a phone with Project Tango for under $500 at CES this year, and new chips from companies like Movidius are giving mobile devices the ability to see and understand the world around them.
I got a sense for the role 360 video will play in VR firsthand while trying to introduce this new technology to my parents. Their first experience with virtual reality arrived on their doorstep, a Google Cardboard packaged along with the paper edition of The New York Times. They are retirees, both in their 60s, at a healthy remove from the cutting edge of modern technology. My mom finally ditched her flip phone only a year ago. My dad still uses AOL for his email. My parents were willing to give virtual reality a spin, but only with me there as a guide.
I walked them through setup and cued up the video, but once it was on, they had no trouble interacting with the 360-degree experience. Watching them smile, laugh, whoop, and cringe, it was clear they felt the magic that has made so many excited about the revitalized VR industry. Later I offered to let them try a game on the Samsung Gear VR, but they weren’t interested in the slightest. Neither ever managed to learn so much as a NES controller, the thought of being immersed in a video game was overwhelming.
This is the big, broad audience YouTube is after with Live 360. Mohan puts it this way: "We leave the real technology choices to our users. You could enjoy it with your phone as they are, but we have enabled integration with Cardboard. If the video is shot in virtual reality mode with depth and spatial sound, that enhances it further." The goal is to deliver something that will feel like magic to the biggest audience possible. YouTube, says Mohan, is "trying to find a way to do this at scale, so we are democratizing this innovation, and making it easily accessible."