clock menu more-arrow no yes mobile

Filed under:

Sony’s wild AR hologram tech continues to get better and better

New, 13 comments

A closer and more realized AR alternative to smart glasses

Photo by Nick Statt / The Verge

Sony has made it a point to come to SXSW, the annual Austin-based tech and culture meet-up, every year with a warehouse full of weird gadgets, demos, games, and other interactive experiences. This year was no different, as Sony opened the doors yesterday on the Wow Factory, its name for the wide-ranging exhibit that blends art and technology borne from its experimental, Japan-based Future Lab program. The experiences in the Wow Factory tend to center on Sony’s display tech, specifically its advances in projectors that ultimately seem to have manifested as a pricey consumer product called the Xperia Touch.

But Sony hasn’t stopped pushing the limits of the tech. The core premise is that with a mix of smart sensors that perform depth detection and motion tracking with a high-quality light source, you can create the closest thing we have today to interactive holograms. The projectors create objects out of light that typically exist on a flat plane either in front of the projector or below on a tabletop. You can interact with these virtual objects using your hands because the projector’s software is able to recognize and track your movements. Effectively, Sony has figured out a way to make augmented reality without requiring you wear bulky goggles or goofy smart glasses.

Going one step further, Sony has designed custom demos that make use of real-world objects. Sony returned to Austin this year with a collaborate music game that combines four of its projectors into a single cohesive system. With small 3D models of instruments, including a miniature saxophone and a piano, users can work together to play a series of songs by directing spotlights to each instrument. The small 3D-printed models are recognized by the software and come to life under the projectors’ light, while other sensors track your finger motions as you move the spotlight around the table.

The demo is not at all practical, because it requires custom software and custom props. And no consumer would ever spend many thousands of dollars to outfit a table with four of Sony’s prototype projectors just to pull off silly games and tech proof-of-concepts like this. But it is a genuinely impressive demonstration, as each object placed under the projectors’ light and within range of the system’s sensors is brought to life in a way that looks and feels like the closest manifestation of software in the real world.

it’s also a great example of how to take an alternative approach to AR. Something like this is both more accessible and can be experienced collectively, without requiring everybody wear a pair of smart glasses, a VR-style helmet, or even a compatible smartphone with the requisite software. Sony’s approach here is akin to a hologram — it exists physically as light in a 3D space that everyone can see and interact with.

Sony has been working on this tech for a while. It’s mostly a marketing stunt to showcase its experimental hardware, but over the years, we’ve seen the full breadth of what this tech allows. We saw Lewis Carroll’s Alice’s Adventures in Wonderland jump off the page and interact with physical objects like a teacup and a deck of cards in 2016, and last year, Sony built an architectural demo to show the enterprise use cases of its projector tech, as a standard block of wood was transformed into a top-down scale model of a home.

This year, Sony engineers took the lessons the company picked up with the Xperia Touch and its prior demos to develop a three-person virtual hockey game. The custom circular table is equipped with a standard projector with its new image IMX382 image sensors to track the puck and paddles, while the projector creates a virtual interface that reacts to your physical movements.

We don’t know whether this tech will ever turn into a viable mainstream consumer product — the mini-projector Sony sells now capable of running these AR-style hologram demos costs about $1,700. And without a true reason to own one or develop applications for it, it’ll never take off in the way AR apps on iOS and Android can, thanks to software frameworks like Apple’s ARKit and Google’s ARCore. But if Sony does find a way to commercialize this tech, it could pave the way for a unique and novel way to create immersive, collaborate AR experiences that can be deployed using everyday objects and on something as ordinary as a kitchen table. That’s exciting, if it ever does leave the quirky demo phase it exists here in Austin.