Microsoft's HoloLens is the kind of thing that science fiction has been preparing us for for decades. From the moment I saw it, I knew exactly what I wanted it to do: project a heads-up display around real life, let me play with imaginary toys and sculpt in holograms, produce weird optical illusions with its magical power to rewrite reality. Unsurprisingly, these are expectations that no product in 2015 could possibly live up to. Within carefully managed conditions, HoloLens is one of the most amazing pieces of tech I've seen. But for all its potential brilliance, it's still struggling hard — and failing — to break down the walls between fantasy and reality.
I wasn't able to see HoloLens in January, but from what I can tell, the hardware has gotten some major polishing since then. There are no wires, and no separate computer. The whole thing is now wrapped in a small and light (relatively speaking) pair of goggles, including a set of miniature speakers and a "holographic processing unit" that powers its tracking capabilities.
HoloLens now actually looks like the thing in Microsoft's concept videosWe were still warned that the prototypes might break if we weren't gentle, and getting a comfortable fit takes some work. When someone else helped me, the HoloLens rested effortlessly around my head; when I tried for myself, it either gave me a headache or kept slipping out of place. Nor has the tight secrecy around it relaxed. Microsoft made us leave behind any electronics, including cameras, and one of the company's PR reps referenced a "background check" while I was trying to sign up for a demo. But externally, it's caught up to the device in Microsoft's promotional videos.
Everything I'd heard about HoloLens before trying it sounded like, essentially, sorcery. Without seeing it, it's hard to know how to take someone telling you that Microsoft can put a castle on a coffee table. But once you turn it on, it makes a kind of sense. HoloLens feels like the world's brightest, highest-quality projector, which it sort of is, except that it's projecting onto your eyes instead of a wall. The first thing I saw was a colorful, oversized origami set, hanging in mid-air. When I passed my hand through it, I could dimly see the outline through the hologram, but that didn't make it feel any less solid. At one point, with a larger projection, I didn't notice someone standing right in front of me.
Microsoft and Trimble's HoloLens architecture program.
Add in HoloLens' sensors and audio, and this can become downright uncanny. The origami demo, for example, included two paper balls that would drop onto a notebook and slowly roll off its edge. In the most primitive version, they fell straight down and disappeared into the ether. When HoloLens began mapping out my surroundings, the balls would thud off the imaginary notebook and roll around the floor, bouncing lightly off tables and couches. There were so many pieces to the illusion that I could almost imagine they were real, especially because there's absolutely no lag. Images can occasionally drift, but it's often possible to walk completely around them without thinking of the fact that a computer is constantly reading your position, calculating the placement of a virtual object, and projecting it onto your retinas.
Microsoft still doesn't have gestures figured out — your major options are "air taps" or voice commands — but it's created demos that give you an impressive sense of control. An architecture program lets you use a mouse to manipulate a 3D model while it's sitting on a table in front of you. Holo Studio, a hands-off demo from January that's now playable, combines voice commands like "rotate" or "airbrush" with a simple point-and-click design interface. It feels a little like the augmented reality version of MS Paint, and that's great.
The HoloLens' amazing illusion only really works with small objectsBut HoloLens only feels natural when you're not handling anything much bigger than a basketball. It produces a magic square the size of a large TV screen, and the moment something slips outside, it disappears. It's possible to imagine that a small object has just dipped out of sight, but for a larger one, you either have to step quite a ways back or content yourself with just seeing pieces of it in the center of your vision. It shatters the illusion, and it looks very little like the amazing whole-world illusions of Microsoft's videos. Even a heads-up display becomes less useful once your peripheral and near-peripheral vision is off-limits. And a couple of Microsoft's ideas clearly just seem meant for virtual, not augmented, reality. You could drop into a hidden world in the origami demo or look around a full-sized landscape in the architecture program, but it's hard to piece together what's going on through that little window, especially when you could be looking at the whole thing at once with an Oculus Rift.
People often imagine virtual and augmented reality fusing, but with HoloLens around, the two start seeming very distinct indeed. Its images are astonishingly good, on a level that VR's magnified screens will probably never match. It's smaller than any virtual reality device on the market, partly because it usually doesn't need to power an entire photo-realistic environment. Microsoft has put much more work into building things that people can use, not just things they can see. But it's hard to imagine how Microsoft (or anyone) could get the HoloLens projection system to support a field of view big enough that it can stop being distracting, let alone become immersive on a VR-like scale. As cool as HoloLens can be, it's firmly a product of today, not the future — at least not yet.