Microsoft didn't have a booth or even an official press event at the Consumer Electronics Show this year, but that didn't stop the company from jumping on stage twice. CEO Steve Ballmer joined Qualcomm for its bizarre opening keynote, and more importantly Microsoft’s Chief Technology Strategy Officer, Eric Rudder, joined the Samsung keynote to showcase the IllumiRoom technology. Based on a combination of a Kinect for Windows camera and a projector, IllumiRoom combines the virtual and physical worlds of a TV and living room for true augmented reality.
A mysterious partnership with Samsung emerges
Microsoft claims IllumiRoom is a "proof-of-concept system," but the fit and finish of its promotional video, compared to the low production quality of typical Microsoft Research videos, makes me think otherwise. "The footage shown here is exactly what appeared in our lab without any special effects being added," said Rudder as he introduced the video last week. The demonstration starts with an actor commanding the Xbox to "go big," with a projection showing an Xbox console scanning the room before images are projected onto nearby walls. Microsoft didn't explain why it chose to demo IllumiRoom with Samsung, but the companies have previously worked together on PixelSense tables and the components for Microsoft's Surface tablet. Samsung manufactures a number of Pico projectors and has recently added the technology to its Galaxy Beam smartphone — suggesting it's in a position to offer the tech at a low-cost for the consumer market.
IllumiRoom lines up with previously leaked Kinect 2 info
Redmond's fascination with augmented reality and Kinect has been well documented. A number of Microsoft Research projects, including Kinect Fusion, Beamatron, Holodesk, Holodeck, and Digits, all show what's possible if Microsoft invests in augmented reality, but most of the projects seem a lot further out than IllumiRoom. A lack of a published research paper, which typically occurs when Microsoft unveils a project, and promised details planned for Chi 2013 in late April means the full unveiling is likely planned just a month before E3 — a date where Microsoft is widely expected to unveil its next Xbox console.
A new Xbox has long been rumored for 2013, but sources familiar with Microsoft's Xbox plans have suggested to The Verge that the company may be preparing an early unveil of some future Xbox plans ahead of a main event at E3. The timing of IllumiRoom would suggest this is part of a larger plan for the future of Xbox. Previously leaked details on Kinect 2 have suggested it will introduce a "LiveWall" that uses a projection unit to "allow game environments to be projected nearly 360 degrees around the user." This information lines up with the IllumiRoom project.
Microsoft is fascinated with augmented reality, especially for Kinect
In a recent visit to Microsoft's Research team in Cambridge, I met with researchers that had contributed to Xbox Kinect. A focus on augmented reality and a no-touch future was obvious, but it also felt clear to me that, like Kinect, Microsoft was using its Research arm to discover and then develop technologies it would use for the future of Xbox. During the visit, Microsoft's Shahram Izadi played a video from Sony that imagined a future immersive experience of video gaming. Afterwards, he claimed that the next big thing is going to be around augmented reality. "It is about drawing the physical world into the user interface and it's about the user interface coming out into the real world. It's about these two worlds colliding and merging together."
Sony's example is obviously extreme, but Microsoft's IllumiRoom tease feels like it's the future of Xbox. We might just have witnessed what the next Xbox is capable of without Microsoft officially unveiling it. Project Natal, the codename for Kinect, was unveiled at E3 in 2009 — over a year before its debut and it caught the gaming industry by surprise. E3 will be all about the next-generation and I fully expect to see Microsoft focus on augmented reality whether it's ready to ship this year or not.