Facebook partners with RED to develop a high-end, professional VR camera

Illustration by James Bareham / The Verge

Facebook is partnering with camera maker RED to develop a professional-grade virtual reality camera system that can capture high-resolution imagery in so-called 6DoF, or six degrees of freedom, which allows it to be viewed and explored in real time within virtual reality. The rig, which doesn’t yet have pricing or release date info available, is a culmination of Facebook’s internal efforts on its Surround 360 platform. RED’s upcoming VR camera will now be the one Facebook suggests to filmmakers and other creators who want to make the most high-fidelity, immersive entertainment and art. The news was announced today at Facebook’s F8 developer conference.

“A year ago we set out to find a hardware partner to help us deliver this technology,” says Brian Cabral, a director of engineering at Facebook specializing in computational photography. Facebook needed not just “the image quality we wanted,” but also the workflow that would let creators easily capture necessary photography and video on set, and be able to transfer that to the necessary software that would then stitch it together and help create VR-ready experiences, or what Cabral calls a “photon to photon” experience.

For the last two years, Facebook has been basically advising the camera industry on what such a camera would look like. The Surround 360 launch, back in 2016, involved launching an open-source spec sheet and hardware design document for a device with a 17-camera array for VR capture. The idea was to help camera makers, and other companies with the industrial and consumer tech manufacturing experience, actually physically produce a product, which those companies could then sell and in turn help more filmmakers and creators produce VR content.

Facebook’s original Surround 360 prototype.
Photo by Nick Statt / The Verge

Facebook, at the time, didn’t want to put its name on a device or sell it directly to consumers. Rather, the company wanted to jumpstart VR creation so consumers actually had content to consume if they bought an Oculus Rift headset or wanted to access 360-video posts or other social video products on mobile. A year later, the Surround 360 team released an updated VR camera design with a 24-camera array, as well as a lighter six-camera array model. Those cameras were capable of capturing 8K video and capturing content in 6DoF, which greatly increases the immersive quality of VR video by letting viewers move naturally as if in a completely virtual world.

“We’ve taken all of our learnings over the past two years and all of our cameras and algorithm development to... inform a lot of our decisions on both our partnerships and the design of the cameras,” Cabral says. “We’ve also listened to a lot of people who use our prototypes and others to incorporate all the learnings in the industry to shoot with VR cams. A lot of the feedback we get is how easy to use it on set.”

Now, with its partnership with RED, it seems Facebook has finally found a company with a reputation in Hollywood and indie filmmaking circles for its technical mastery to push high-end VR video closer to the mainstream. Cabral says the camera is designed for professional storytellers. “The idea is to enable the best storytellers and give them the best equipment,” he adds.

Facebook and RED don’t yet have an image of working a prototype or render of the product to share, and RED will decide at a later date how expensive the camera will be. But given that RED cameras typically scale from $5,000 to $15,000 to even $50,000 — the company last fall announced an $80,000 8K model — we can only assume this camera will be very expensive. Still, for Facebook, the more high-end VR content that’s out in the world, the more attractive its Oculus and video-centric Facebook Watch platforms become.

Comments

But given that RED cameras typically scale from $5,000 to $15,000 to even $50,000 — the company last fall announced an $80,000 8K model

Most of the time, you only get the body of the camera at those prices. RED makes you buy all of the overpriced accessories yourself.

You haven’t a clue what you’re talking about

Because Facebook doesn’t have enough information on me, let’s have them manage my own image in VR, too.

Photogrammetry sucks, these VR cameras really need to come with attached or integrated solid state LIDAR or VCSEL sensors.

What does this have to do with photogrammetry?

He’s referring to this part of the article:

"virtual reality camera system that can capture high-resolution imagery in so-called 6DoF, or six degrees of freedom, which allows it to be viewed and explored in real time within virtual reality."

He’s assuming, and I’m guessing he’s right, that they will be capturing enough data through photography (using spaced lenses) to play it all back in a pseudo-3d environment so you can essentially lean your head left or right, forward or back, and the video won’t go with you (like it does now in normal 360 degree video).

I agree with him. Photogrammetry hasn’t been gracefully utilized yet in VR that I’ve seen. Look into Matterport. They have a camera that uses infrared and does a pretty great job of 3d scanning a room. The problem is it takes several scans from multiple locations. Not great if you’re trying to film a story like Facebook seems to point to.

Not photogrammetry or scanning like what Matterport uses. Lens spacing allows depth information to be captured, allowing slight movement in the video.

We still have yet to see a single example of compelling VR film.

It’s been years since the first VR film shorts showed up. Nobody has made anything of note.

It’s fun to see the film progress in 360 but this all seems like a lot of money to be throwing at a thing we haven’t seen a competent proof-of-concept from.

Why are you assuming its obvious use is to tell a story?

No matter what you do in VR, (working, gaming, socializing, learning) the experience is greatly improved with a compelling and appropriate environment. Those environments could be 360 photos, videos, or of course 3D renderings.

I love working in VR when the work doesn’t require a lot of small text (I’ve done a fair bit of video editing in VR). That will change when resolution improves. At that point I can manage any computer work, in any environment I want.

If you work better in a noisy environment, imaging choosing a looping 360 video shot in the middle of time square, and putting your virtual desk in the middle of it. Not into noise? Drop your desk into the bottom of the ocean, a perfectly beautiful park or maybe you’d prefer to sit in the Oval Office.

When you hold a virtual meeting, you’ll see the other participants around you, and they will see you. But you’ll all be seeing your own chosen environments.

A beautiful High Rez video capture of 10 mins of looping footage would make a wildly compelling place to work or learn in VR.

I want one of these new cameras. Badly. But if I had one, I wouldn’t try to replace flat screen storytelling, I’d put a creative team together to start scheduling travel, and capturing historical spaces, exciting events and breathtaking vistas. Because eventually, there will be millions of people working in VR, and they will all want a new place to work when they log in each day.

Sounds like I just need $100k.

So the cart has been parked here for a while and the horse doesn’t even have a prototype?

View All Comments
Back to top ↑