HoloLens is turning into the technological equivalent of a tragic hero — the kind whose extraordinary talent isn’t enough to overcome a terrible personal flaw. At this spring’s Build conference, Microsoft showed off beautifully clever demos that were undercut at every point by the complete inability to see them through the headset’s narrow field of view. This is more obvious than ever at E3, where you’re not looking at business or research tools. You’re supposed to just be having fun.
This week, the HoloLens team officially unveiled a gorgeous augmented reality version of Minecraft, which Microsoft bought last year. Watching the stage demo was incredible — it’s a whole pixelated diorama, projected onto a coffee table. It’s also testing the future of what it calls mixed-reality gaming with Project X-Ray, a tech demo where you defend your living room from mechanical aliens. These are both very good ideas. They are also ahead of their time... or at least ahead of the technology they’ll need to really work.
Minecraft is like a step toward holographic Legos
HoloLens Minecraft’s real draw is its augmented reality miniatures, but the setup actually involves two parts, one of which uses an Xbox gamepad, and the other, HoloLens’ simple gesture and voice controls. Players project a traditional flat screen rendering of Minecraft on the side of a wall, where it sits like a holographic TV. Then, they render the entire environment as a 3D model on something like a coffee table, incidentally creating one of Microsoft’s most striking E3 images in the process.
Microsoft hasn’t created a whole new interface for the game, so actual building takes place on the screen. The table model is more like a maquette, letting you examine the world from a bird’s-eye view. You can interact with the environment to a limited extent — I could zoom around the environment, call down lightning bolts, and leave signs by staring at an area and telling Minecraft to mark it. You can even tap and drag the whole world to see multiple layers.
The ultimate HoloLens Minecraft might be more like a set of augmented reality Legos that you could play with right from the table, but this isn’t a bad start. It’s also a completely adorable one, once you start looking inside tiny buildings and watching the smoke come out of their tiny chimneys while tiny cows mill around outside until being hit by a tiny lightning bolt. But as you get close to the table — as you’ll inevitably want to do — parts of the world start clipping off, and Microsoft’s tapping gesture isn’t completely reliable. The diorama ought to make Minecraft feel more natural, but playing with it starts to feel like a struggle.
Cut out everything but the center of this press shot, and you've got the actual HoloLens experience.
The real problems, though, start when games try to rewrite the world in less compact and self-contained ways — even if those make for some of the most fascinating experiences. Project X-Ray sounds exactly like a setup that was created dozens of times in old phone-based "augmented reality" games: a video game enemy gets projected into the real world, and you have to shoot it until it dies. One of our few glimpses of Magic Leap involves this very idea. Microsoft, though, has reinvented it with HoloLens’ excellent room-scanning technology. Your alien enemies don’t just appear, they crawl out of realistic-looking holes in the real walls. They tunnel under sheetrock, leaving trails of sparks behind them. They force you to engage with physical space, dodging around the room to avoid lasers and get a better bead on them. It’s not a full game by a long shot, but it’s an example of how natural and un-gimmicky Microsoft can make the concept feel.
You can't write off HoloLens as a failed prototype
To get the full effect, though, it would need to cover much more of the world than HoloLens allows. The headset’s field of view is around the size of a big-screen TV. Step back, and it’s more than big enough to see one of the holes that aliens have punched through your wall. Once they start crawling or flying, though, they’ll quickly disappear from sight, leaving you with a glowing arrow indicating where to look. At some points, this feels like a valid gameplay decision — I tried to imagine that I was actually in a space suit with a tiny window for my face, confused and frightened. Mostly, though, it just reminds you that none of this is real, and it removes the sense of spatial awareness that’s one of augmented reality’s great theoretical strengths.
It’s a shame, because the holograms are uncannily real. The objects nearly always look solid, and they stay in place almost perfectly. The voice recognition is fairly perceptive. The headset’s even comfortable. If every part of HoloLens were a little bit frustrating, or its demos offered something no one wanted, it would be easy to write it off as a failed prototype. Instead, it’s like Microsoft created an amazing alternate universe and walled it away.
On one hand, this means Microsoft only has one clear and serious problem to solve. On the other, we don’t know how difficult it would be to fix it without compromising the lightweight design. It’s also hard to gauge how much of a problem Microsoft thinks the field of view actually is, although it’s received plenty of feedback over the past months. And right now, it’s showing off games whose worst enemy is their own platform.