How Lucasfilm made Star Wars’ first official VR lightsaber battle

"It’s like VR was almost made for holding lightsabers," says Lucasfilm chief technology officer Rob Bredow, who introduced the project to a small group of journalists at this year’s Game Developers Conference in San Francisco. With the lightsaber, you’ll deflect the stormtroopers’ incoming (but strangely slow) blaster fire back at them with deadly results, holding them off until the Falcon can make its escape.

According to ILMxLab creative director John Gaeta, Trials on Tatooine was the result of the team’s conversations while filming The Force Awakens. "We were basically having a sort of fetishistic relationship with the Millennium Falcon last year, so that was actually one of the starting points, oddly — that we wanted to feel its actual scale and sort of what it was like to actually be there with it," he says. "It was that simple in the beginning." From there, they tried to imagine how someone might want to interact with the world, whether through tinkering, fighting, or both.

Contrary to some speculation when its trailer leaked last week, Trials on Tatooine isn’t a game or full-fledged interactive experience. It’s the equivalent of a single movie scene, with no firm plans to develop it further or send it out for release. This puts it in the company of several Star Wars-themed prototypes that ILMxLab has developed in the past few years. In one, viewers used a tablet to watch a scene of stormtroopers hunting for R2-D2 and C-3PO on Tatooine, but with the option of switching locations and perspectives to uncover different threads of the plot. Another, released for Google Cardboard on the Star Wars mobile app, offered hints about the plot of The Force Awakens in the lead-up to the movie’s December release.

As a piece of the Star Wars mythos, though, it’s unusually ambitious. Until now, the group tended to use virtual and augmented reality to flesh out well-known parts of the existing Star Wars films, offering extra scenes or an alternative perspective. But Trials on Tatooine is a rare glimpse into the still-enigmatic time period between the original trilogy and its latest sequel, taking place at some point before a cataclysmic betrayal that sets up The Force Awakens. Small as it is, it’s telling the kind of story that might otherwise appear in the Star Wars universe’s many extended universe books, comics, and TV shows.

"[Trials on Tatooine] was just what it was called — an experiment," says Lucasfilm executive planner Vicky Dobbs Beck. "We learned a lot about first-person storytelling — about how you balance interaction with story — and that'll be valuable information as we conceive our future experiences."

Gaeta hints that this could be exactly where ILMxLab’s VR experiments are headed. "Normally, what we would be inclined to do would be to have you dropping into something for which there was a lot of context going in," he says. "If we weren't doing that, for certain we would want to serialize this, right? We would want to build over time until such a point where things started to really gel." This was the format of ILMxLab’s pre-Force Awakens VR teaser, Jakku Spy. But like many other VR developers, the team may still be determining how to pull it off for a more standalone project. "It's a difficult thing to do short-form anything in VR, for the same reason it's hard to have enough exposition in [short-form] cinema," says Gaeta. "So no, that is not at this time part of a series. But that's the approach that we would take in general."

Despite these difficulties, the Star Wars canon seems almost uniquely suited for these short-form projects. It’s so large and detailed that there are endless corners and crevices to be filled with new narrative, but everything eventually ties back to a handful of easily recognizable concepts, characters, and set pieces — it’s a universe that’s both expansive and insular. "Before we have any [movie] scripts developed, there is a really serious drill-down on what the trajectory of the universe, and the number of really important character threads, should be across time," says Gaeta. "So how does Force evolve across time? How does the war evolve across time?" The films might remain at the center, but almost any medium can tell a story that runs along that same trajectory.

Trials on Tatooine still feels truncated, partly because it sits so squarely between VR film and game. As a cinematic action sequence, it flows fine, framing a classic Jedi-versus-stormtroopers battle with Solo’s fast-paced patter. But using the lightsaber requires developing legitimate skill, since you’ve got to catch blaster bolts at just the right angle to send them ricocheting back. By the time I’d started to get the hang of it, the scene was over, and replaying it would mean sitting through several minutes of lightsaber-less dialog I’d already heard. There’s a tension between the story ILMxLab wants to sweep you through — about the Millennium Falcon, its crew, and a Tatooine-bound Force adept — and the personal journey of mastering an awe-inspiring (if fictitious) skill, preventing either one from reaching its full potential.

That’s the kind of thing, though, that ILMxLab’s experiments are designed to detect and eventually solve. The question is how long that might take — and what virtual reality will even look like by the time it happens. Over the course of a few years, we’ve gone from simple stationary headsets to room-sized tracking systems like the Vive and sophisticated augmented reality prototypes like Microsoft’s HoloLens. To Gaeta, though, this won’t affect the core of what they’re doing. "Tech is going to change, but people are always going to want to have certain types of interactions," he says; they’ll want to look at a world and have the world react to their presence.

And as technology advances, users will be able to do this in a form that feels more and more natural. "Every time you look at these pictures from the past of VR or AR, you laugh, because the stuff looks like crazy clunky stuff. Like, wearing a box on your head is not exactly very fashion-forward," says Gaeta. "It inevitably shrinks back to human, and that's where it will keep going." Just as cameras evolved from capturing stationary footage to mimicking almost anything a human can do, virtual reality could become increasingly easier to mistake for reality. "Holding hard plastic controllers is the most foreign thing you can do — you really want to be in there literally expressing with your body, you want to be lost inside of it," he says, referring to the Vive remote that currently serves as both lightsaber and hand. "The tech will become more powerful in order to get more human into it."