Camera maker Lytro drew our attention three years ago with what it called "light field" photography. Instead of focusing at a specific depth, its camera would capture everything in range of its finder, so you could do things like pick a focus after taking the picture. It was a fascinating technology, but one that appealed mostly to camera buffs. Earlier this year, though, the company cut its workforce, raised $50 million in funding, and began focusing on how light fields could be used in virtual reality. Today, it's releasing its first camera rig for VR video: the Lytro Immerge.
Lytro previously added VR headset support for still images, but the Immerge isn't just a traditional camera that takes video. Like the Jaunt Neo system, it's a ring of cameras that capture almost an entire sphere, combined with a software setup that lets filmmakers blend them all together. Actually, we're talking about several separate rings, which Lytro CEO Jason Rosenthal says contain "multiple hundreds" of cameras and sensors. Like Lytro's still cameras, the Immerge will work alongside a software system that processes it for various VR headsets. Unlike those cameras, the Immerge captures enough data that Lytro also created a portable server that can store an hour of video.
Very simple 360-degree video is like a flat cylinder wrapped around a headset user's field of view. Unlike a VR video game, there's no three-dimensional space to move around, even if we're just talking about leaning a few inches to one side. As Rosenthal puts it, "it's like your whole environment is thumbtacked to you," and existing attempts at parallax video are clumsy. But getting a huge range of focal dimensions could help improve the experience. "What if you could densely capture all the light rays flowing in that direction, within that space, and play them back with high frame rate and high fidelity?" he asks. Having detailed depth information also opens the door to adding more realistic computer-generated props, or even mapping the real image onto a more game-like 3D environment.
Jaunt has also referenced using light fields in its camera technology. Lytro apparently appreciates Jaunt's work but is (unsurprisingly) more confident in its own product. "The whole Jaunt camera would be equivalent to a fraction of one of the [Immerge] rings that we're talking about," claims Rosenthal. "They're using 'light field' more as a marketing term than from a technical perspective." Jaunt also primarily considers itself a VR film studio, not a camera company.
"It's like your whole environment is thumbtacked to you."
In case it wasn't already clear, the Immerge is a high-end camera for studios like Vrse, which produces virtual reality video for the likes of Vice and The New York Times. (Vrse co-founder and CTO Aaron Koblin, among other VR filmmakers, lent Lytro some support in a statement, saying that "light field technology is probably going to be at the core of most narrative VR.") That means that when a prototype version comes out in the first quarter of 2016, it'll cost "multiple hundreds of thousands of dollars" and is intended for rental. Rosenthal also says this will let Lytro get feedback and upgrade the cameras more quickly — he suggests we could see upgrades quarterly.
With headsets like the Rift, Vive, and lower-end Gear VR on the horizon, cinema is one of the clearest non-gaming uses of virtual reality. Filmmakers have produced both independent work and advertising tie-ins — usually under 10 minutes in length — for the medium, and a Jaunt-sponsored Sundance Institute VR film residency was announced earlier this week.