Skip to main content

Oculus acquires 3D mapping company Surreal Vision to turn reality into a video game

Oculus acquires 3D mapping company Surreal Vision to turn reality into a video game

Share this story

If you buy something from a Verge link, Vox Media may earn a commission. See our ethics statement.

Virtual reality company Oculus has acquired Surreal Vision, a UK company whose software can map and recreate the real world in a virtual one. Surreal Vision grew out of its three co-founders' Ph.D. research at Imperial College London; the team will now move to Oculus' lab in Redmond, Washington.

This is the latest of several acquisitions by Oculus, which was itself purchased by Facebook in March of 2014. After the Facebook buyout, it brought on the design team that helped create the Xbox 360 controller and Kinect, followed by motion-tracking company Nimble VR. In a blog post today, Surreal Vision gave us an idea of what it will be doing (emphasis added.)

From the human point of view, the world is constantly in motion. As we move around, our eyes dart about the scene and the rich dynamical nature of the scene's contents come flooding in. We're able to make sense of those changing signals to produce a coherent understanding of the world we live in, which we effortlessly navigate and interact with. Over the past three decades, a great deal of work in computer vision has attempted to mimic human-class perceptual capabilities using color and depth cameras.

At Surreal Vision, we are overhauling state-of-the-art 3D scene reconstruction algorithms to provide a rich, up-to-date model of everything in the environment including people and their interactions with each other. We're developing breakthrough techniques to capture, interpret, manage, analyze, and finally re-project in real-time a model of reality back to the user in a way that feels real, creating a new, mixed reality that brings together the virtual and real worlds.

Ultimately, these technologies will lead to VR and AR systems that can be used in any condition, day or night, indoors or outdoors. They will open the door to true telepresence, where people can visit anyone, anywhere.

Much progress has been made toward this future, but significant challenges remain. For virtual reality, the accuracy and quality of the continuously updating 3D reconstruction must be near flawless, which is a requirement almost no other modern computer vision problem faces. When we cross these seminal thresholds, users will perceive the virtual world as truly real - and that is the experience we're driving toward.

For an example of what that might look like, we can look to SLAM++, a virtual mapping system developed by Surreal Vision co-founders. In the demo video above, a depth-sensing camera scans a room, and a program analyzes its contents, matching it against a pre-rendered database of objects like chairs and tables. From there, digital creations can appear to interact with real-world ones. (Back in 2013, when the demo below was filmed, that meant having virtual human beings stand up from real chairs and dance to "Gangnam Style.")

Being able to apply game physics to real space is central to augmented reality projects like Microsoft HoloLens, but so far, Oculus has limited real-world interactions in its own worlds. Now that it's got a finished product officially coming next year, though, it's promised to show us more ways to use VR. Whatever the team ends up working on at Oculus, one of the unique aspects of SLAM++ is that it doesn't just find the shape of a thing, it makes guesses at what it is.

Augmented reality often involves adding bells and whistles to the real world. But something like Surreal Vision's work could be equally important for, as the team mentions, telepresence. It's how you could buy virtual tickets to an NBA game and sit convincingly on the bleachers, or sip from a coffee cup during a virtual meeting. And above all, it could dovetail with work on motion control — one of the central problems that VR and AR enthusiasts are still trying to crack.