Microsoft Tribe

Forums

What is "Project Fortaleza" (Kinect glasses)?

2

Reading over The Verge's article about the Xbox 720 document leak, one thing struck me more than anything else. Though the promise of a 6 core Xbox, 3D Kinect, and cloud gaming are exciting, the real game changer revealed in the document is a point that was barely mentioned, Project Fortaleza. The basic idea behind Project Fortaleza, as interpreted by Tom Warren, is a gaming-focused competitor to Google's Project Glass. In the words of the original leaked document (which has since been taken offline by Microsoft's lawyers), Project Fortaleza is a "breakthrough heads up and hands-free device." So what does that mean?

When the Kinect was first revealed in June 2009 as Project Natal, a seed of an idea was planted in the back of my mind that has grown over the past three years. The seed was this: the sensor powering the Kinect, made by an Israeli company called PrimeSense, could be used to create a science fiction-like augmented reality HUD (Heads Up Display) system like the one used by Robert Downey Jr. in Iron Man. Iron-man-2-viral-webcam-augment-site_medium

Iron Man's Heads Up Display via cdn.screenrant.com

How would this work? Basically, as you look around a room, the infrared sensor from the Kinect/PrimeSense camera would be used to build a live 3D model. Virtual objects could be placed in this model; for an example, a virtual TV could be placed on an empty wall. Whenever somebody wearing those glasses were to walk into that room, the 3D model would be matched to the previously made 3D model and the TV would appear.

But this goes beyond simply placing virtual objects. Imagine playing a zombe video game. As you walk around your house, zombes would pop up from behind couches and from underneath beds that have been 3D mapped (in live-time) by the Kinect/PrimeSense camera. To kill the zombes, you have to find real objects around your house to attack them with. In the opposite direction, these glasses could be a game-changer in the business world. Imagine that you are an architect and you're sitting around a conference table with a client. You think that it would be a great idea to put a giant water slide in the middle of the office building you're designing, but your client just isn't buying it. To try and win them over, you use your glasses to display a live 3D model in the middle of the conference table. Using a worker-simulation app, you show your client how much time employees would save if they used a water slide to go downstairs. When you're showing them the 3D model without the water slides, your client realizes that workers have a long way to walk to get to the copy room. Using your glasses, you are able to move the copy room on the spot and demonstrate to your clients the improved work flow. Though your client does not agree to have the water slide built, the collaboration through the HUD glasses has lead to a more efficiently-designed building.

So what evidence do we have that Microsoft is actually building these glasses? Let's start with the leaked name, Project Fortaleza. Microsoft traditionally code names projects after cities, and Fortaleza is a city in Brazil. In addition, Fortaleza is the site of an ambitious Brazilian government metro-building project, but I'm guessing that that is just a coincidence. So it appears that the code name checks out, and this could be an actual Microsoft project, not just the dreams of a fan boy in his parent's basement. (The fact that the Power Point was taken offline by Microsoft's lawyers also suggests that it was real).

480px-montagem_fortaleza_medium

Fortaleza, Brazil via upload.wikimedia.org

So do these "Fortaleza" glasses actually match what I described above? They appear to. In the picture representing the Wifi version of the glasses (dated 2014), two teenagers are playing a video game against a "hologram" of a cowboy in their living room. In addition, the specs of Kinect Two describe a three-dimensional image feed which would likely be required in the sensor on these glasses for them to work as I described. Though I can't fully discern what the women is doing in the 2015-dated image for 4G Fortaleza glasses, it appears that she is viewing some sort of virtual screen while shopping. This sugests that Microsoft sees the potential of Fortaleza glasses beyond the sphere of gaming.

Xboxglasses_medium

via assets.sbnation.com

Unfortunately, the document is dated 2010, and a lot can change in four years. As we all painfully know, Microsoft has a tendency of killing off their most ambitious, forward-thinking ideas before they reach market. Hopefully, Microsoft sees enough potential in Project Fortaleza to keep it alive and give it the funding to reach market. If they do, Microsoft could revolutionize the gaming, business, education, and consumer industries and once again become the undisputed leader in technology.