Skip to main content

I tried Magic Leap and saw a flawed glimpse of mixed reality’s amazing potential

Photography by James Bareham | Video by Tom Connors & Becca Farsace

Share this story

When you write about augmented reality headsets, you’re supposed to start by describing something impossible — like a pastel dinosaur stomping its feet in a quiet office space in Florida. This dinosaur is made of fist-sized blocks that look like candy, and the office belongs to Magic Leap, a mysterious startup that’s been working in near-total secrecy for seven years. I should clarify that the dinosaur also isn’t real. It exists only in the lenses of the Magic Leap One, a pair of goggles that Magic Leap hopes will replace phones, computers, and every other high-tech screen in our lives.

The whimsical anecdote setup is supposed to emphasize how well the Magic Leap One tricked my mind into believing this impossible thing existed, which is what I’d hoped would happen last month when Magic Leap invited me to its headquarters. But it just didn’t happen.

In reality, the dinosaur I see through the Magic Leap One looks genuinely three-dimensional, but pieces start getting cut off when I approach it. When a man walks behind it, I can see him slightly. My headset doesn’t account for relative distance, so it’s impossible for someone to walk in front of the dinosaur, no matter how close they are. It’s still a fascinating, wonderful illusion — maybe the best I’ve seen in one of these headsets, and far cooler than watching an AR model through an iPhone screen. But it’s not the kind of revolutionary (or downright magical) advance that Magic Leap has teased for years. It’s a better version of a thing I’ve tried before, and that thing is still very much a work in progress.

Based on an afternoon with Magic Leap, the Magic Leap One Creator Edition — which ships in the US today for $2,295 — is a functional, thoughtfully designed headset with some very real advantages over competitors like the Microsoft HoloLens. But it doesn’t seem like a satisfying computing device or a radical step forward for mixed reality. Magic Leap’s vision is a compelling alternative to that of Silicon Valley’s tech giants. But there’s a baffling disconnect between its vast resources and parts of its actual product. I genuinely believe Magic Leap has given me a glimpse of the future of computing, but it might take a long time to reach that future, and I’m not sure Magic Leap will be the company that gets there first.

Magic Leap calls itself a “spatial computing” company, but it produces what most people call augmented or mixed reality experiences: hologram-like objects projected into three-dimensional space. Modern smartphones offer a primitive version of mixed reality, and headsets like Microsoft HoloLens offer a more advanced version for industrial and professional use. Magic Leap has a more ambitious goal: it’s building futuristic mixed reality glasses for everyday computing, hoping to beat bigger companies like Apple or Facebook to market. So far, it’s raised over $2.3 billion to realize its vision, with major investors that range from Google to JPMorgan.

The company isn’t just promoting a headset. Since its founding in 2011, Magic Leap has cultivated an enigmatic image that CEO Rony Abovitz compares to the black monolith from 2001: A Space Odyssey — “where it’s anything you want it to be,” he says. Abovitz has claimed that Magic Leap’s hardware will “transcend what can be contained in a physical product.” He announced the company with a 2012 TedX talk in which he donned a full space suit and spoke for 30 seconds. Today, he won’t even confirm it was him in the suit.

“Our whole thing with Magic Leap One is, we want people to realize this is what computing should look like.”

The Magic Leap One Creator Edition is aimed at artists and developers, but Abovitz stresses that it’s a “full-blown, working consumer-grade product,” not a prototype. AT&T will even offer demos to customers in some of its stores later this year. “We think it’s at the border of being practical for everybody,” says Abovitz. “Our whole thing with Magic Leap One is, we want people to realize this is what computing should look like — not [laptops], not TVs, not phones.”

The Magic Leap One is a three-piece system that includes a headset called Lightwear, a small wearable computer called the Lightpack, and a handheld controller. The headset is studded with tracking cameras for mapping your environment, as well as inward-facing eye-tracking cameras. The darkened lenses are inset with small glass waveguides, which Magic Leap calls “photonics chips.” These chips are manufactured at Magic Leap headquarters, a former Motorola factory. Abovitz says Magic Leap can comfortably produce tens of millions of chips on-site, while the rest of the headset is manufactured by a third party whose name and location he resolutely refuses to discuss.

Lightwear eschews the visor-like style of most mixed reality headsets, favoring a cyberpunk-y design with big round lenses, which lead designer Gary Natsume says are supposed to evoke the “universal sign” of glasses. “If you can start drawing two circles, and everyone says ‘Oh, that’s Magic Leap,’ that’s our goal,” he says. The system has a headphone jack, but by default, it pipes audio through small built-in speakers near your ears.

The headset looks far from utilitarian — it’s like something a hacker would wear in a Shadowrun larp. But against all odds, it’s surprisingly comfortable. You put it on by stretching an expandable back, then settling it lightly over your head. Buyers can pick from two sizes, which offer the same eyepieces with different head strap sizes and interpupillary distances. Swappable nose and forehead rests make smaller adjustments possible. You can’t wear normal glasses with the Magic Leap One, but you can order prescription lenses that clip magnetically into the headset.

Possibly the only computer that comes with a reality button

Part of the comfort factor is the Magic Leap One’s relative lightness, since the bulkiest electronics are offloaded into the Lightpack. The little puck computer is permanently wired to the headset, and it includes an Nvidia Tegra X2 chipset, 8GB of memory, 128GB of storage, and a battery with up to three hours of use time. The battery is partly detached to create a slot in the center of the device, so you can clip it onto your pocket; if you don’t have pockets, you can also snap it into a thin shoulder strap and wear it that way.

Otherwise, the Lightpack is a smooth gray disk with a USB-C charging port and buttons for power and volume, as well as a universal pause button cleverly labeled “Reality.” Natsume says the Lightpack is specifically slightly smaller than a CD-ROM, so people won’t mistake it for a CD player. Given the Magic Leap One’s mildly ‘90s-retro look, that seems like a legitimate concern.

The Magic Leap One is designed for an eclectic range of inputs. It supports some third-party controllers, and the Lightwear cameras track limited hand motion, although I only tried this feature briefly. Abovitz even says Magic Leap will release a “biomarker toolkit,” which will let apps extrapolate breathing patterns, tone of voice, pupil changes, and pulse rate from microphone and eye-tracking camera data.

But the default interface is a single remote-style controller informally called the “totem,” which includes a front trigger, a bumper button above the trigger, a round trackpad on the top, and a tiny home button behind the trackpad. It looks a lot like the controller of the Oculus Go or Samsung Gear VR, except that it’s fully tracked by the headset cameras, so it offers a full range of motion. It’s not quite like having a virtual hand, but it’s a versatile system nonetheless.

Like every mixed reality company, Magic Leap eventually wants to make a normal-looking pair of glasses that can be worn everywhere. For now, the headset is only guaranteed to work indoors, and it includes Bluetooth and Wi-Fi antennas, but no mobile data options. AT&T has already committed to selling a future version with wireless data plans, though, and Abovitz says you can use the current version “at your own risk” outside. “We wanted to teach people how to begin to live a life like this. You don’t suddenly want people running across the street,” he says.

I didn’t take the Magic Leap One outside, and I’m not sure how well it would function under bright sunlight or near dark. I tried it in a series of spacious, well-lit rooms filled with furniture that provided clear hard edges for tracking. I played demos built by Magic Leap’s huge team of developers to show off the headset’s full range of capabilities, with an employee on hand to answer my questions and guide me through the experiences. I saw the Magic Leap One at what any reasonable person might assume was its absolute peak performance. And in light of all that, I still left worried.

Magic Leap is best known for its hardware, but the roughly 1,500-person company has a large software team as well, and the Magic Leap One has a functional operating system and starting app suite. Its Linux-based Lumen OS appears as a series of balls floating in mid-air, and the headset ships with a web browser called Helio; a “social suite” with a holographic chat system; an app store called Magic Leap World; an image gallery; a system for pinning and watching virtual screens; and a demo of Dr. Grordbort’s Invaders, a steampunk shooter from New Zealand special effects studio Weta Workshop.

Magic Leap World offers a few more experiences, including an NBA mixed reality app preview; an art app called Create; and an interactive musical experience called Tonandi, made in partnership with the band Sigur Rós. These apps cover some of mixed reality’s clearest use cases, but they’re compromised by the headset’s basic technical limitations.

Field of view is a big problem for headsets, and the Magic Leap One is no exception

The Magic Leap One’s field of view is constantly distracting. Field of view is a huge problem for mixed reality headsets, which can generally just project images into a moderately sized rectangle in front of you, leaving the rest of the world bare. Magic Leap has improved on Microsoft’s HoloLens in this respect — it’s got a 50-degree diagonal field of view, which works out to a rectangle that’s around 45 percent bigger. But it’s not nearly enough to look around the world normally. Moderately sized objects were cut off if I got too close, and full-room scenes appeared only in patches.

The overall image quality, meanwhile, felt similar to HoloLens. Objects looked three-dimensional, but ethereal. Edges glowed slightly, text was a little fuzzy, and some objects appeared slightly transparent. Tracking was generally good, but objects occasionally shifted or jittered. A few times, animated objects seized up altogether, which might have been an issue with tracking, Lightpack performance, or something else. Magic Leap theoretically features multiple focal planes that let your eyes focus more normally than with other mixed reality headsets. But the images weren’t realistic enough for me to judge how well that was working.

All of this, to be clear, is still very, very impressive compared to most mixed reality headsets. Combined with the comfort factor, it makes Magic Leap one of the best (if not the best) pieces of mixed reality hardware I’ve seen. But after all of Magic Leap’s descriptions of its unique hyper-advanced light field technology, it didn’t feel categorically different from something like HoloLens — which was released two years ago, and has a second generation on the horizon. I’m not convinced Magic Leap’s photonics chip is practically that different from other mixed reality waveguides, or that Magic Leap is doing something other companies couldn’t replicate.

I’m also not sure how much the Lightpack’s performance will limit experience size, complexity, and spatial responsiveness. The Magic Leap One is supposed to have fairly sophisticated tracking options. I could put my palm out to block projectiles in Dr. Grordbort’s Invaders, and I could walk around a room to create a rough mesh of my surroundings, then do things like bounce a virtual ball against a couch. But for now, the headset seemingly ignores all non-static objects besides hands. I was told that it could potentially track people moving around the dinosaur I mentioned earlier, but that doing so would require too much power — so the option won’t be available in the first release.

Some of Magic Leaps first wearable prototypes

Revolutionary computing devices don’t have to be perfect, as long as they do something even a little exciting — and Magic Leap is more than a little exciting. Abovitz compares the Magic Leap One to the first Macintosh computer. “It didn’t have full color, it didn’t have all the stuff. But the whole world just opened up for me,” he says. “It’s got all these infinite possibilities.”

But instead of showcasing the strength of its possibilities, my Magic Leap One app demos kept highlighting the weaknesses of its technology. I could imagine replacing my television with a virtual screen, but not one that clips in half when I’m not staring straight at it. I kept forgetting where I’d placed small virtual objects in a room. Full-room experiences, like the beautiful underwater seascape of Tonandi, always felt clearly artificial. The issue wasn’t just technical limits, it was apps that didn’t seem designed to work well within those limits.

Magic Leap seems poised to make more interesting experiences than the ones I saw

The apps also generally just weren’t very novel or interesting. Most were modest riffs on existing HoloLens or phone-based mixed reality experiences, like a generic toybox of 3D props or a Wayfair webapp for visualizing furniture in a room. I didn’t see the ambitious-sounding projects that Rolling Stone described in a glowing profile last year — like a full-fledged narrative sci-fi scene, a virtual comic book, a theme park experience, and a hyper-realistic virtual woman.

My marquee experience was Dr. Grordbort’s Invaders, which Rolling Stone says has been in development for five years and has a 55-person team. (For context, that’s triple or quadruple the size of many prominent indie game studios.) But it was still an abbreviated shooting gallery with simpler gameplay than Magic Leap’s original concept demo or Microsoft’s Project X-Ray HoloLens demo, both of which appeared over three years ago.

If anybody is in a position to make groundbreaking, intuitive, and entertaining mixed reality experiences, it should be Magic Leap. Abovitz and his co-founder Sam Miller originally built their hardware to support an expansive sci-fi project called Hour Blue — Abovitz compares their path to a movie studio making an X-wing fighter prop for a Star Wars movie, then going into aerospace because the thing can actually fly.

The musical Magic Leap experience Tonandi.
The musical Magic Leap experience Tonandi.
Image: Magic Leap
Image: Magic Leap

Hour Blue is his personal side project now, but Magic Leap still has a powerhouse creative team, including Snow Crash author Neal Stephenson, The 7th Guest creator Graeme Devin, and “bullet time” co-inventor John Gaeta. It’s also got partnerships with several artists and development studios, including a deal with Disney’s Lucasfilm.

So unless Magic Leap is deliberately holding any big projects for a consumer release, I’m not sure what its internal studios and partners have been doing with several years and virtually unlimited funding, and why it wouldn’t showcase more of their work during the Magic Leap One’s big debut. Magic Leap doesn’t seem insulated from competition by sheer technological superiority. Its best bet might be convincing people that it’s offering the most interesting vision for our mixed reality future. I wish I’d seen more signs of that future in my demos — because based on Abovitz’s descriptions, it’s pretty distinctive.

The simple fact that Magic Leap is building a mixed reality headset doesn’t seem as singular now as it did in 2014, when Google led the company’s first big funding round. Companies like Microsoft, Meta, and Avegant are making similar devices, and Apple, Google, and Facebook are heavily promoting phone-based mixed reality, with not-so-veiled plans to release glasses in the future.

And although none of these companies have tried to make mixed reality glasses mainstream, Magic Leap isn’t trying to do that right away, either. “We can push a little bit, but at some point you want to move with society, listen to people, I’m not ‘go fast and break things,’” he says, referencing Facebook’s infamous early slogan.

The mostly failed Google Glass headset from 2013 showed how badly people can react to augmented reality headsets — it came off as creepy, invasive, and elitist, the province of a few oblivious nerds who can unilaterally record or remake their environment. Magic Leap plans to use its funding to stay afloat while it waits for social norms to evolve. That strategy seems risky, since unlike a larger company, Magic Leap doesn’t have another product to fall back on. But during a period of backlash against Facebook and other large companies, it might give Magic Leap an advantage.

“We want people to operate in a certain decent cultural way.”

“Our whole thing is, we want to be hyper-protective of the user,” Abovitz says. He wants to package all the data users generate into a user-owned repository called a “lifestream,” which people can keep private or explicitly license to companies. He says Magic Leap users can do whatever they want in private spaces, but that the company will take a conservative approach to apps and publicly distributed art, which should be “G to PG-13” with no porn or ultraviolence.

It’s a hands-on approach to platform-building with an explicitly prosocial bent — Abovitz imagines releasing a “young person’s primer to being a good Magic Leaper” to engrain mixed reality etiquette in kids.

Magic Leap’s biggest plans would change our entire relationship with computers in some very weird ways. It’s apparently working on dual AI assistants: a simple robotic creature for performing low-level tasks, and a separate human-like entity that you’d treat as an equal, to the point that it will leave the room if you’re rude. “If you break terms and conditions, and you’re really rude to an AI, our general thought process is you might lose access and you might have to regain access,” he speculates. “Almost like if you’re bad in a chat room or bad in Wikipedia. We want people to operate in a certain decent cultural way within our overall ecology.”

But it’s easy to make big promises when you have no users. Everyone I met at Magic Leap emphasized that the AI, the “biomarkers,” the lifestream, and other experimental ideas were long-term projects. As it stands, Magic Leap seems extremely vulnerable to being upstaged by a bigger company — Apple’s increasing investment in AR isn’t particularly secret, after all — and that’s if the world turns out to be ready for mixed reality at all.

Magic Leap has set impossible standards for itself over the past seven years, and as the Magic Leap One ships, Abovitz wants to reset some of the hype. “It’s all extremes, and I just feel like, ‘Hey, let’s all come back to the center, let’s take a breath,” he says. “But it’s probably a lot cooler than most people think.”

Whether or not it’s cooler than most people think, the Magic Leap One is cooler than the vast majority of mixed reality in 2018. But it still seems a long way from realizing the promise of the medium — and it hasn’t shown that it can bridge that gap.