When Microsoft first unveiled its IllumiRoom technology at CES, it did so in an unusual way. The software giant traditionally reveals its research projects in short, scrappy videos accompanied by research papers and a lot of technical jargon. IllumiRoom was different. The chief technology strategy officer at Microsoft, Eric Rudder, joined the Samsung keynote to showcase a video of an augmented reality projection unit designed to extend gaming onto nearby walls. The demo showed off a concept that appeared finely tuned, using what felt like a marketing video.
"IllumiRoom envisions a next-generation gaming console..."
Microsoft described IllumiRoom as a "proof-of-concept system" and promised more information at a CHI 2013 event that starts this week. While the research team behind IllumiRoom will reveal the project's technical background tomorrow, a fresh preview video (30 seconds long) and a new five-minute demonstration (see below) have both provided some additional information. "IllumiRoom envisions a next-generation gaming console with a projector that sits on your coffee table and surrounds your television with projected light," says the preview video of the technology.
"Oh, my TV got bigger."
Microsoft has remained fairly quiet about its next-generation Xbox plans and IllumiRoom. The Verge spoke to Microsoft's Eric Rudder at TechForum in early March about IllumiRoom. Rudder and the Xbox team were quick to stress it's simply a proof-of-concept project, but they did share some basic information about how it works. "The LED is projecting for a frame and then turning it off and Kinect IR is reading those layovers," said Rudder. "Then the Kinect turns off for a frame and then the projector goes on and then Kinect will go back and forth." Rudder described the technology as "incredibly sophisticated," but that the experience is "Oh, my TV got bigger, and I can project on anything because the surface doesn't need to be flat."
The current prototype uses a Kinect for Windows camera alongside a projector to transfer content onto walls. Kinect captures the color scheme and geometry of the room while the projector displays all of the visuals around the TV. Microsoft says its IllumiRoom system is self-calibrating and designed to work in any living room. There are a variety of ways in which IllumiRoom can add value to games, including by extending the content, highlighting the edges of games, or enhancing select game elements like weapons fire or explosions. Another method involves displaying markers, either for specific elements in a game or other characters in a multiplayer game.
IllumiRoom can make objects appear to fall out of your TV
IllumiRoom can also saturate room colors, making them appear black and white, or highlight the edges of a room. A radial wobble feature makes a room feel like it's responding to a game by popping and moving as the game is played. Grid, starfield, and snow visuals react to the game and display on nearby walls. Microsoft is also working on lighting effects to capture motion from the game and display it within a living room. This augmented reality extends to game objects falling out of the screen and onto a living room floor.
Shortly after Microsoft's first unveiling of IllumiRoom, we speculated that it was connected to the company's next-generation Xbox and could be similar to Project Natal—the original codename for Kinect. Microsoft originally unveiled Kinect at E3 2009 before shipping the accessory over a year later. Microsoft's hint that IllumiRoom is linked to the company's next-generation Xbox could mean we'll see a lot more at the upcoming May 21st event or at E3 in early June.
Update: Microsoft has now released a full video demonstration of IllumiRoom and a PDF research paper. We have updated the article to include the new details.