clock menu more-arrow no yes
Illustration by William Joel / The Verge

Filed under:

Fading Light: the story of Magic Leap’s lost mixed reality magnum opus

One of the startup’s most exciting projects is finished, but will anyone see it?

In April 2020, the weird and ambitious startup Magic Leap cut its workforce in half and delayed plans to take mixed reality glasses mainstream. The company had a wealth of ideas about how ordinary people might use its hardware, which overlays virtual images on reality. But after years of development, many were still prototypes or tech demos. Magic Leap was done with consumers for the near future, and it didn’t seem to be leaving much behind.

Inside the company, though, a few dozen developers were building what they describe as one of Magic Leap’s most exciting projects. It’s called The Last Light: an interactive story about a young woman dealing with the death of her grandmother, designed to show the storytelling potential of mixed reality. And crucially, its creators say it’s finished — but they aren’t sure if anyone will ever see it.

Photo by James Bareham / The Verge

Magic Leap is one of the biggest, best-funded players in mixed (or augmented) reality, a blanket term for tech that blends the physical and virtual worlds. The Florida-based startup received more than $2 billion in funding and was known for recruiting high-profile talent to chase futuristic applications of its tech. One early hire was Snow Crash author Neal Stephenson, who, until recently, ran a research lab in Seattle. Another was Magic Leap “chief games wizard” Graeme Devine, co-creator of classic games like The 7th Guest. While much of Magic Leap’s work was secret, CEO Rony Abovitz touted concepts like a sophisticated virtual assistant named Mica and a city-sized holographic overlay called a Magicverse.

But alongside these grand sci-fi ideas, a now-gutted division was working on more immediately practical projects. The team, called Magic Leap Studios, designed apps for Magic Leap’s earliest headsets. The Last Light was its most ambitious project, meant to prove that first-generation mixed reality could still tell powerful stories — and to keep Magic Leap’s creative side alive as the company shifted to business customers. According to current and former employees, it was just weeks away from a successful debut. Then the coronavirus pandemic brought the world to a crashing halt — and The Last Light, along with much of Magic Leap, crashed, too.

Studios began as a general-purpose content division. “Creative [design] was always part of the inception story and myth of Magic Leap,” says Anastasia Devana, the team’s former sound director. “That’s why Studios existed.” Early work was done under two of Magic Leap’s celebrity hires: Devine, who ran the studio, and the author Austin Grossman, who was apparently helping design a mixed reality dungeon-crawler called Bitforce. But Bitforce was discontinued and Devine left Florida within months, stepping back from day-to-day operations.

The ultimate result, according to Studios members, was positive. The roughly 70-person team shifted toward more manageable, practical, and collaborative projects — primarily Create, an art tool that shipped on the Magic Leap One in 2018. “There were always egos, but not the kind of egos that will collapse the whole project because someone’s vision is so grandiose and they just want people to get it done,” says lead artist Mouhsine Adnani.

After Create, Studios wanted to build something more ambitious. It settled on a multiplayer puzzle game codenamed “Gemini,” which one studio member compared to the hit indie game Monument Valley. But the multiplayer component felt too complicated, and the puzzles never clicked. Neither delivered what some team members really wanted: a narrative experience that would touch people who didn’t care about Magic Leap.

Narrative was part of Magic Leap’s DNA from the beginning. Abovitz originally founded the company not as a hardware startup, but as a home for a sweeping transmedia fantasy epic. But Studios’ past attempts hadn’t worked out. “Story was always this golden chalice we could never quite grab,” says lead producer Bryan Jury. The more they looked at Gemini, though, the less compelling its gameplay seemed. Then, creative head Jeremy Vanhoozer brought in a short story about a girl and her grandmother, and The Last Light was born.

The Last Light is about a young woman named Kayah. After Kayah’s grandmother dies, Kayah returns home and picks up the pieces of her life. As she explores the house and the world around it, old memories resurface and new details about their relationship emerge.

Many mixed reality experiences — including Magic Leap’s best-known prototypes — try to realistically blend virtual objects into real rooms. The Last Light was different. Kayah’s story takes place on an island-like floating stage, which can display anything from a modest living room to a glittering waterfall. As she moves through the story, memories begin appearing on the real walls of a viewer’s room, creating the illusion of small inset dioramas.

The design was inspired by architectural maquettes and stage plays as well as a short virtual reality animation called Age of Sail. Viewers can trigger The Last Light’s scenes at their own pace, but they can’t directly control its characters or story, only how they move around to experience it. “It doesn’t need to be flashy or crazy, that’s not what we want you to focus on,” says Dave Shumway, The Last Light’s audio lead. “We just want you to focus on this story being told in a unique way.” The roughly 40-minute experience was far longer than most mixed reality art, and the attention to art, music, and voice acting made it more than a simple animation.

The Last Light was also deeply personal. Vanhoozer’s script was inspired by his real relationship with his grandparents. Devana lost her own grandmother during its production. “It deals with the loss of a family member, so I think a lot of people could relate to that. Certainly I could relate to it,” she says. “It was great to know that it was possible to do on this platform. It was possible to create something meaningful that genuinely touched people, not just a little tech demo.”

Studios saw The Last Light as mixed reality at its best: an artistic experience that played with physical space in a way no other medium could, but could offer something more than novelty. And at first, that felt emblematic of Magic Leap’s overall vision. The company was working with well-known game studios like Insomniac on meditative games like Seedling, where players restore a dying ecosystem by growing holographic plants. “When we started it felt like a really confident, comfortable thing for us to do after we’d done Create,” says Shumway. But at Magic Leap, things were about to change.

Photo by James Bareham / The Verge

Magic Leap was a young company with a sweeping mission and a lot of money — and that could translate into a chaotic workplace. Studios employees describe a company that offered incredible creative freedom, but also frequent distractions and competition between divisions. “It was filled with a whole bunch of kingdoms,” says Jury. “They tried to do everything, and nobody was ever able to rein themselves in.” At one point, he says, the Create team literally hid behind curtains to avoid fielding requests for new projects.

But as Studios was building The Last Light, Magic Leap began heading in a more conventional direction. The startup was a massive outlier in the world of mixed reality, where nearly every viable company caters to a niche audience of businesses, research institutions, or the military. Magic Leap initially seemed committed to bucking that trend — it named its Magic Leap One Creator Edition to attract artists and entertainers, and Abovitz contended that the hardware was “at the border of being practical for everybody.”

Gradually, though, it started treating consumer glasses as a longer-term goal. “There was a shift in Magic Leap’s vision,” says Adnani. In December 2019, the company renamed its “Creator Edition” to the more official-sounding Magic Leap 1 and launched a special program for enterprise customers. And at Studios, Shumway says, a new line of thinking took hold. “It became, well, ‘What we were approved to do lined up with what we thought Magic Leap was going to do. Now that doesn’t necessarily seem to be the case anymore... but no one’s telling us to stop.’”

Studios kept working, though, apparently with support from executives — including Abovitz. And around the start of 2020, it scored a huge win: The Last Light was accepted into the SXSW Film Festival’s Virtual Cinema Competition.

Festivals are one of few ways for cinematic mixed and virtual reality to find a substantial audience and press coverage, especially if they require rare and expensive hardware. The Sundance Film Festival has showcased Magic Leap-powered experiences for the past two years, but The Last Light would have been its longest project as well as the first developed entirely by Magic Leap instead of an outside artist or organization.

A successful showing at SXSW would set The Last Light up for months of future festival appearances and a relatively mainstream audience, not just a handful of early adopters with headsets. And it gave Studios credibility. “That was a big deal for us,” says Adnani. “It felt like we had to prove our existence and why we should be there, but it gave us some space to tell them: hey, look, we’re making some really cool stuff. And people love it.” Even if Magic Leap started focusing on enterprise, Studios could point to a track record of building impressive software — then stick around until it made another bid for the mainstream. “We knew that if we could get to SXSW it would be a little bit of a lifeline,” says Jury.

The team created a shortened version of The Last Light for SXSW and began finishing their final cut, largely a matter of editing audio and recording voice actors’ final lines. The situation at Magic Leap seemed under control. The rest of the country was another matter.

The United States marked its first COVID-19 infection in January 2020 and its first death a few weeks later in Washington state. SXSW held out even as other conventions and conferences were canceled, but as artists and exhibitors dropped out one by one and case numbers ticked higher, holding a 30,000-person event became harder and harder to justify. On March 6th, SXSW organizers officially canceled the show a week before it was due to start, as the official tally of US cases cleared 200 for the first time.

It didn’t feel like a disaster to everyone at first, just a setback. “There was no indication from the company that anything was going to happen at that point,” Shumway says. “So we didn’t feel like, oh man that’s it, that’s the shot, we had it and it’s gone. That was never part of it. Because we always anticipated either SXSW would come back in a few months, or if that didn’t happen, that we’d just go into the next one.”

But as they prepped applications to more festivals, Jury wasn’t as optimistic. “It just didn’t feel like it was heading in that direction,” he says. “I remember having a pretty heavy conversation with Jeremy [Vanhoozer] about it — and his response to me was, well, sometimes you just have to pretend that you’re going to be doing it.”

The weeks passed, and 2020 looked increasingly barren. The Tribeca and Cannes film festivals, two of the biggest venues for mixed reality, were postponed indefinitely. Florida Gov. Ron DeSantis issued a stay-at-home order on April 1st, locking down nonessential businesses within the state. Magic Leap employees, including Studios staff, took their headsets and started working from home.

Some employees weren’t worried about their jobs, and they felt like Studios played a crucial role at Magic Leap. The thinking was that “we’re the ones making content,” as Adnani puts it. “Without us, there’s nothing on the device.” Others, however, started hearing rumors about serious changes within the company. They had to move fast, they decided — or The Last Light might end up on Magic Leap’s scrapheap. “We actually were checking in until 2 or 3 in the morning the day before the layoffs,” says Shumway. “By that point we had an idea it wasn’t good, and whatever needed to get in probably needed to get in right then, because we may not have access the next day.”

Just as they wrapped the experience, Magic Leap laid off half of its roughly 2,000-employee workforce, including nearly everyone on the Last Light team. Their work was in the hands of a few remaining Studios members — and a company that no longer seemed to care about it.

Magic Leap 1 Updated Headset

Magic Leap averted a worst-case scenario of more layoffs, closing a $350 million investment that let it keep the doors open. The company is still working on a Magic Leap 2, which it’s said it will release in 2021, but it seems radically transformed. Abovitz announced plans to step down as CEO in May, saying the company needed someone new to “commercialize our focused plan for spatial computing in enterprise,” a far cry from the whimsical language of years past.

The Last Light is apparently in limbo. Its creators say it’s so close to finished that Magic Leap could release it on the headset’s app store, something that was supposed to happen after its festival run anyway. But the company has been cagey about its future. “Last Light is a powerful and emotional work from the extremely talented Magic Leap studios team. We are very proud of this piece and are exploring various options for its release,” a Magic Leap spokesperson told The Verge.

Some employees suggested that Magic Leap might be worried about the optics of releasing a mixed reality art piece during a pivot to business-only mixed reality, or the risk of doing anything that might garner too much attention during its transition. Conversely, if Magic Leap is still hoping for a festival release after the pandemic, launching Last Light online would reduce its appeal to those venues. But it’s not clear when these festivals will come back, and they may change in ways that diminish their mixed reality sections. The next Sundance Film Festival, for instance, will be spread across at least 20 cities — which could be poorly suited for its New Frontier hub, a home for singularly elaborate installation-based virtual and augmented reality. And that’s assuming the pandemic doesn’t keep people leery of sharing headgear.

The Last Light’s creators never expected it to have the reach of a traditional game or movie — that’s the price of building for a brand-new medium. With an app store release, they don’t even expect the kind of audience they’d get from a series of film festivals; one unofficial report suggests Magic Leap only sold around 6,000 headsets in its first six months. Phone-based augmented reality is broadly accessible, but it’s even more clunky and limited than current-generation headsets. And removing the mixed reality element entirely — the interplay between real and virtual space — would strip the experience of its full power.

But no matter how small the audience, former Studios members have pragmatic reasons for wanting it released, in addition to their emotional investments. Magic Leap’s layoffs left hundreds of people looking for new jobs, although many since have found work at games studios or big tech companies like Apple. If The Last Light is buried, its creators will lose the chance to show off their best work — a perennial issue in the larger games industry, where huge teams work on secret projects that are regularly scrapped after months or years of development. The project officially exists only as a short trailer on YouTube, which cuts together small clips of the story in a traditional video format.

To some, The Last Light also shows that Magic Leap isn’t an inevitable failure or an elaborate scam — as its harshest critics have dubbed it, “the Theranos of augmented reality.” The company may have done too much, too soon, with too little discipline. But it could still produce experiences that pushed the boundaries of its medium. “Magic Leap was an overinflated hype machine, but we were genuinely doing some great work at Studios,” says former senior designer Javier Busto.

Despite their frustrations, the employees I spoke to have fond memories of the company. “I didn’t join Magic Leap for the technology, which is probably weird. I joined because there were a lot of really interesting, challenging problems to solve,” says Devana. She still believes in the medium of mixed reality. “I do think it’s the future. I’m just not sure how soon it will come.”

The question is whether Magic Leap has the resources to shape that medium, especially with some remaining employees reportedly considering departures, according to Jury. “They’re going to lose everyone who knows how to make software on this hardware,” he says — not just him and other laid-off staff. “I think a really small, focused company that focuses on one or two really specific things could work. But they’re going to lose so much talent.”

It’s difficult to tell how much COVID-19 changed Magic Leap’s direction. The company had been appealing to professional clients and reportedly seeking acquisition for months before the pandemic. Barring a stupendous technical breakthrough, its Magic Leap 2 probably won’t be small or cheap enough for a mass audience. Even if Studios had survived a transition to enterprise, employees may not have wanted to stay. “I don’t think any one of us joined Magic Leap to make a weather app or a calculator or an Excel program,” says Adnani. “We joined it because we wanted to make some really cool stuff.”

But wherever the overall company was headed, The Last Light’s sudden demise seems driven almost completely by the coronavirus. Had SXSW gone as planned, it’s doubtful Magic Leap would have pulled an ambitious experience out of a popular festival even if it caused confusion over its brand identity — especially an experience that Studios members say its CEO loved. Rather than a single company’s failure, it feels like a painfully familiar story for many people living through the pandemic: a dream that looked finally ready to become reality, until reality broke down around it.

For now, former Studios staff are still wondering if The Last Light will ever leave Magic Leap’s servers. “Everyone worked really hard on this,” says Adnani. “I wish they would just press the button and release it.”

Entertainment

Fast & Furious can’t resist that joke about turning it up to 11, so you get one last movie

Tech

The Hummer is back as a 350-mile range ‘electric supertruck’ that can drive diagonally

Tech

Catch Rep. Alexandria Ocasio-Cortez live tonight on Twitch with HasanAbi and Pokimane

View all stories in Tech