Florida-based startup Magic Leap became infamous for hyping itself as a revolutionary company whose vision for augmented and mixed reality would transform the worlds of computing, communication, and entertainment — whenever it finally decided to show its hyper-secretive technology to the public, that is. The company set expectations high, but when it finally shipped its first headset, the Magic Leap One Creator Editor, the consensus was that some of those expectations hadn’t been met. Magic Leap hadn’t magically solved AR’s biggest challenges, it turned out. Like the Microsoft HoloLens before it, Magic Leap’s headset was a glimpse at a compelling, but far from mature, technology that simply couldn’t deliver on the grand vision founder Rony Abovitz had laid out for augmented and mixed reality — or what Magic Leap calls “spatial computing.”
Last week in Los Angeles, the company held its inaugural developers’ conference, dubbed L.E.A.P. Con. (The acronym stands for “learn, engage, accelerate, and program.”) In many ways, a developers’ conference feels like the perfect environment to discuss a nascent technology like Magic Leap. This kind of gathering focuses heavily on potential — what creators and executives hope will one day be possible — while allowing a healthy dose of demonstrations and guidance about what’s actually possible today. The conference was a snapshot of the current state of the company, and it was hard to not walk away invigorated by the potential of AR and MR — even though the timeline for anyone to deliver on that potential is still up in the air.
Noble vision or tech utopianism?
From the beginning of the conference’s keynote address, it was clear that Abovitz wouldn’t be shying away from the enthusiastic hyperbole that’s been a hallmark of Magic Leap from the very beginning. “Today our world feels divided,” he said. “It feels broken.” From there, he wound up a pitch about how Magic Leap’s spatial computing is a new medium, and can therefore be built to avoid the problems with trolling, abuse, and lack of representation that plague other media, from the web to television.
That’s a noble idea, and Abovitz is correct that designing platforms with a clear-eyed understanding of how people have weaponized products like Facebook and Twitter could result in media that better protect users. But delivering on that will require specifics that simply weren’t addressed. That makes statements like “spatial computing can be a safe haven and a creative space to include all who respect each other, all who want to build new worlds” come off as Star Trek-inspired utopianism — particularly given that the discussion is about some far-off possible future medium, rather than a functioning platform that exists today.
The real look at the company’s big-picture vision came later, when Abovitz took the stage with Magic Leap chief futurist (and science fiction author) Neal Stephenson, and the company’s senior vice president of creative strategy, John Gaeta. The trio broke down the company’s vision for the “Magicverse” — the virtual destination guests step into when using Magic Leap hardware — which would expand far beyond the single-use applications available for the headset today. They hope the Magicverse will one day include vast, city-scale projects (the company is in early discussions about some of these initiatives already, they teased without explanation) where different virtual “layers” could offer a number of different experiences atop a single, real-world location. One layer could simply be a digital re-skin of a physical city, turning a downtown business district into a futuristic science fiction landscape, for example. Other layers could offer entertainment options, while still others would be dedicated to communications and public utilities.
The details about these different layers were sparse, of course. It’s easy to criticize the company for talking up such blue-sky thinking, when the modern-day technology is so limited. But that kind of approach is also vital if a company wants to think well beyond present technological possibilities. Other companies may not speak so openly about their grand visions, to avoid setting expectations that can’t be met in the short term. But for better or worse, restraint has never been Abovitz’s style.
A strong proof of concept that demonstrates the potential power of the medium
In the demonstrations of what the Magic Leap One can actually do, the strengths and weaknesses of today’s technology came sharply into focus. Weta Workshop and Magic Leap’s Dr. Grordbort’s Invaders was the flagship gaming demo at the conference, dropping participants into a shoot-’em-up scenario where the targets — robotic alien creatures — emerged from portals overlaid on the actual walls of the room. It’s an immediately accessible piece of fun, and a strong proof of concept that can demonstrate to a general audience how magical it can feel for a real environment to be digitally transformed around them. As one portal opened, I crept up on it, peering into an alternate dimension that extended far beyond what had previously just been an ordinary wall. When robots blasted energy weapons at me, I wound up dropping to the floor to take cover behind a coffee table.
Like virtual reality, augmented and mixed reality both suffer from technological limitations that prevent imagery from being completely photorealistic. But while VR wraps audiences into completely artificial environments, AR and MR are able to leverage and interact with the actual physical location you’re standing in. That fundamental difference made it that much easier for me to buy into the illusion of Dr. Grordbort’s Invaders, even with Magic Leap’s current limitations.
However, with robots coming at me from multiple walls, the headset’s limited 50-degree field of view was hard to miss, and if I got too close to the portal on a given wall, the digital imagery would falter and vanish until I took a step back. But the game was nevertheless good at using small visual and audio cues to direct player focus — enough so that I almost always engaged with shooting the approaching robots with the Magic Leap controller, rather than twisting my head to make sure I could see them.
Insomniac Games’ Seedling was a simpler demo, letting players cultivate an alien plant on their own table. The gameplay was simple, and the use of the Magic Leap controller to switch between a watering can and pruning shears didn’t provide the most intuitive experience, but it was still a cute distraction, with the colorful plant growing larger as tiny alien creatures danced and hovered around it. What stood out most in Seedling, however, was a pre-game sequence where I had to flip through a digital magazine that appeared on the table in front of me. The hand-tracking was imperfect, but I eventually managed to flip through the magazine just by pinching the corners of the magazine. Again, it was a reminder that when the technology advances further, it will be intuitive and impressive — it’s just not there yet.
Other demos included a piece in which a motion-captured performer from the Royal Shakespeare Company recited a speech from Shakespeare’s As You Like It on a table in front of me. I was able to walk up close to the performer, or to either side, and it gave the performance a sense of intimacy that was markedly different from film, TV, or even live theater.
Unfortunately Magic Leap didn’t let me eat the Porgs
On the more commercial side of things was Star Wars: Project Porg. An experiment put together by ILMxLAB, the project is an experiment to see how the technology could be used to create what essentially amounts to persistent digital pets. A handful of Porgs roamed around the demo space, and I was able to pick them up with the controller and put them on different surfaces, or interact with them using digital treats and toy balls. The Porgs didn’t look completely solid — something my colleague Adi Robertson encountered in several Magic Leap demos earlier this year — but they did respond to my actions. They would back up if I got too close, and if I decided to repeatedly drop virtual toys on their heads, they would bleat and get more and more upset. (Following Chewbacca’s lead and eating one unfortunately wasn’t an option.)
The version of the experience that will be released to the public in December will allow the creatures to roam the user’s living room, but the conference demo also featured interaction with smart home devices, allowing a tabletop fan to ruffle a Porg’s feathers, or causing the creatures to respond to music being played in the room. It’s another demonstration of how — once again, one day — the headset will be able to integrate with real environments in any number of ways.
But the most arresting demonstration of the conference for me didn’t have anything to do with space creatures, killer robots, or alien life forms. It was a highly detailed digital avatar named Mica. In the demonstration, a riff on Marina Abramović’s The Artist Is Present, I sat down at a wooden table opposite the digital character. With the Magic Leap One tracking my eye movement, Mica was able to maintain eye contact, and her subtle smiles and other facial expressions gave the distinct impression she was an interactive digital being. Mica wasn’t completely responding to what I was saying or doing — though that is the eventual hope — but the demo nevertheless hinted at the ability to establish what felt like a real connection with a digital creation.
It felt more like sitting opposite an actor in a short immersive theater production than taking part in a tech demo, and it was the moment that left me most intrigued about the potential of mixed reality as a medium. The partially occluded digital character, sitting down with me over a table I could feel with my own two hands, grounded the illusion of Mica in a way that was noticeable, with the actual physical reality Magic Leap was projecting upon doing much of the heavy lifting.
I found that simple, non-verbal exchange of gazes impactful in a way few digital experiences are — and one that would be almost comical if attempted with AR through a smartphone screen. Magic Leap has talked about Mica as a potential digital assistant of the future, not unlike Amazon’s Alexa or Apple’s Siri, but that very obvious use case isn’t what intrigued me. Instead, it was how adept this technology is, even in these very early days, at simulating the experience of being next to another human being. The potential there is so vast, stretching across so many use cases and types of experiences, that it’s difficult to fathom. Thinking about it solely in terms of clunky digital assistants seems reductive. In this environment, Abovitz’s blue-sky thinking and outsized claims about what this technology could become seem more inviting, and even believable.
While each of the demos tended to highlight one particular aspect of what Magic Leap might one day be able to do, the specter of what it can’t do was omnipresent. During an afternoon presentation, ILMxLAB’s mixed reality supervisor, Michael Koperwas, outlined the numerous experiments the company has engaged in since beginning its collaboration with Magic Leap several years ago. It was a fascinating look inside the creative process.
At one point, xLAB toyed with the idea of mixed-reality movie-screen extensions that could bring the action off a screen and into the physical space above the audience. It’s also explored the idea of using Star Wars-style holograms to give video chats a new flair. The company considered bringing characters like C-3PO to life, but found that human-sized characters were ultimately a distraction, given that Magic Leap’s limited field of view would clip portions of the characters unless the audience was standing far away. That led to focusing on experiences that could take place in a “story bubble,” or focusing on smaller characters in general.
Magic Leap needs compelling examples and killer apps to become a viable consumer product
Designing around a technology’s limitations isn’t necessarily a bad thing, but it’s worth noting, because so much of the experimental work a company like ILMxLAB is doing is also about finding out what a new technology is best suited to do. Magic Leap has a firm grasp on the utility of future versions of its technology, but seems less certain about what it should be used for today other than experimentation. In the process, a different kind of chicken-and-egg problem begins to present itself: Magic Leap needs compelling examples and killer apps to be a viable consumer product, but the current limitations of the hardware could prevent creators from ever hitting upon those use cases in the first place.
All of which leads back to Rony Abovitz and his grand pronouncements about Magic Leap’s potential. It’s undeniable that the company is working on fascinating technology, and is collaborating with some of the most forward-thinking minds in the world when it comes to entertainment, gaming, and extended reality. Companies like AT&T already see Magic Leap as a massive opportunity — the company is building a 5G network on Magic Leap’s corporate campus to help it develop 5G-enabled applications, and AT&T Communications CEO John Donovan compared the device to the iPhone during the L.E.A.P. Con keynote. There is no question that augmented and mixed reality are incredibly compelling, and the future Abovitz envisions is thrilling.
But Magic Leap’s headset is still in its relative infancy. Talking to developers and attendees at L.E.A.P. Con, it was clear that everyone was enthusiastic about the potential of the device and the AR medium. But it will take time to get there: time for more iterations of the hardware, time for developers and creators to experiment and play, and time for social norms to change to the point where people will feel comfortable walking around in the real world with things around them obscured by AR overlays.
In the meantime, Abovitz’s gift for ambitious hype could be exactly what the company needs to keep developers, investors, and potential business partners excited — even though he’s set expectations the company’s first headset couldn’t meet. With a project as early in its development cycle as Magic Leap, it’s vital to highlight the future everyone involved is working toward. In that sense, the company’s most important products aren’t hardware and games: they’re possibility and promise.