Even by virtual reality’s usual standards, the University of Southern California’s Institute for Creative Technologies is taking things a little over the top.
“You’re gonna see this drone fly up,” warns Todd Richmond, IEEE fellow and director of advanced prototype development. In ICT’s warehouse-like Los Angeles lab, he’s indicating a brand-new miniature quadcopter, which researchers are just starting to put through its paces. From inside a head-mounted display, though, the drone is a hovering sphere in a nearly featureless virtual landscape. I move towards it, balancing the massive headset and trying to keep glowing elastic bands from slipping off my shoes. It drifts away. “It’ll always be two meters from your face, so you can never actually touch it,” he says.
That simple floating ball requires the kind of hardware that’s rarely seen outside science fiction movies. The setup starts with a square patch of tracking cameras above our heads, safely buffered from the plastic storage tubs and old electronics that line the walls. The cameras can tell that I’m chasing a drone (and tell the drone to get away from me) because we’re both covered in LED markers. The quadcopter’s are fixed to its top, and mine are placed all over my body: two strapped onto my feet, two on a pair of fingerless neoprene gloves, one on a large black belt, a few on the large metal horns that jut up from my heavy virtual reality headset. There’s a high-powered computer in a pack on my back. "The one thing we do need to do is — we need a web shooter out of the gloves," jokes Richmond.
The piecemeal prototype feels like an encapsulation of ICT’s mission. Its projects are often crude and conceptual, designed to evoke the possibilities of a technology instead of creating a product. Inside an unmarked and totally unremarkable brick building, the institute showcases things like Project BlueShark, an augmented reality naval control center. And virtual experiences for the Wide5 headset, whose 150-degree field of view puts the Oculus Rift to shame. And the Light Stage, a massive geodesic dome laced with lights and cameras, designed to turn a human being into a perfect 3D copy. It’s not explicitly creating the future, just trying to imagine it.
You may be familiar with the technology, even if the name doesn’t ring a bell. Light Stage’s lifelike body scans, for instance, have ended up in movies like Avatar and The Curious Case of Benjamin Button — the creators won an Academy Award for the technology in 2010. ICT uses realistic virtual characters to let young adults with autism practice job interviews, or to help officers-in-training read interpersonal cues.
The institute was founded in 1999 with a $45 million contract from the US Army — USC described it as "neutral ground for a marriage of Hollywood and high technology." Its mission was to draw on new entertainment tools to build military training simulations, which could in turn help push film, television, and video games forward. (The Army renewed the grant for $100 million in 2004, and the institute still works in partnership with it and other military research departments.)
ICT is particularly known for its work with virtual reality headsets and hyper-realistic digital humans. Among other things, it’s where Albert "Skip" Rizzo developed a VR-based treatment of PTSD for soldiers returning from Iraq and Afghanistan. (It’s also where Full Spectrum Warrior, the video game Rizzo drew from when building his simulated war zones, got its start.) Even amidst the current boom in virtual entertainment, the highly publicized Virtual Iraq/Afghanistan program is still one of VR’s clearest applications.
More recently, ICT was the workplace of Palmer Luckey, co-founder of Oculus and one of the most influential figures in modern VR. In his search for a perfect (or at least adequate) virtual display, Luckey had gotten his hands on the Wide5 headset, co-created by ICT’s Mixed Reality Lab head Mark Bolas. When Luckey got in touch directly to talk about it, Bolas hired him as a technician. Starting in 2011, he worked on ICT projects while independently building his own headset prototypes — which would soon turn into the first Oculus Rift. When I visit the lab, Bolas shows me "Palmer’s desk," a small niche in a conference-sized room whose tables are covered with jumbled electronics.
The Oculus Rift now has its own place in Project BlueShark, a joint ICT-Navy initiative that uses the headset to mimic a futuristic ship’s bridge. As I settle into the BlueShark command center, it’s not hard to imagine it as some dedicated gamer’s labor of love: a plethora of screens arranged around a swivel chair, combined with a VR headset, an extremely simple interface mockup, and yet another pair of LED straps that glow red around my hands. "We struggled somewhat with people misunderestimating what the system was designed to do," says Richmond of BlueShark. "We would put this glass cockpit up, and people would say ‘Oh, that’s a really poor design for a ship interface.’ And our response is, we’re not trying to design that! What we’re doing is building a platform where now you could start to design those things."
ICT builds its own hardware when necessary, but it often repurposes existing products. The Oculus Rift is one example, but others are weirder: one table of simple VR headsets features a Hasbro My3D Viewer toy from 2011, spray-painted olive drab and velcroed to what looks like an old cellphone keyboard. Consequently, its projects have a slightly retro-futuristic feel, even when they’re working with ideas that are years from making it into mainstream tech.
A prime example is the Wide5 head-mounted display itself, which was originally released by Bolas’ company Fakespace in 2006. The Wide5 was named for its field of view, which is 40 to 50 degrees wider than anything Sony, Valve, or Oculus have considered releasing. But it’s also unmistakeably old, a heavy display that sits precariously on my head and leaves gaps that let the real world in.
A new version of the Wide5 could be a genuinely unique modern headset, but upgrading it isn’t high on ICT’s list of priorities. "By the time the private sector catches up, and things start to become commoditized, the lab goes on to the next thing," says Richmond. "We’re at a point where I don’t think that the lab is necessarily looking at doing wider field of view, because industry is going to fix that." ICT would rather use the old Wide5 to show off new material — like a delicate jellyfish that’s especially beautiful at close range, or experimental ways to move in VR.
Soon after my arrival, I strap in for a demonstration of redirected walking, a perceptual trick that slowly guides users in circles while making them think they’re exploring a far larger space. Inside a simulation, a user is tasked with searching a row of simple two-room houses for illegal weapons. But when they turn their back, the rooms change, doors flipping to lead them out in a different direction than they came.
I know all this before the simulation starts, so I carefully examine my surroundings, noting the placement of every piece of furniture. I head through into the back room, preparing myself. Then I turn around. Something seems off as I walk back over the threshold and onto the street outside. But even with the explanation, it’s hard to wrap my head around what just happened. Without one, I’m not even sure I would have noticed.
Richmond talks about the prospect of playing real-life Madden in a small room that feels like a football field, but he reiterates that ICT isn’t just trying to become a game studio or an Oculus competitor. Redirected walking, for example, will be used in training simulators, and motion-tracked drones could be deployed in disaster areas.
He admits, though, that the lines between academia, government, Hollywood, and Silicon Valley are shifting. "Twenty, thirty years ago, the government had the cool technology, and the private sector was lagging behind. Fast-forward to today, the government’s lagging behind and the private sector has all the cool technology," Richmond says. Applying for government funding may no longer be as attractive as working for a private company’s moonshot lab. "What’s the role of academia now?"
That role seems especially precarious when companies come out with products either similar to or directly influenced by ICT’s experiments, especially if those companies get credit for inventing it. Google’s Cardboard headset, for example, was predated by a nearly identical ICT project called FOV2GO. "As long as we’re still able to be funded to work on interesting problems, we’re kind of okay with that. Otherwise we’d be out doing startup companies," says Richmond. "We kind of accept that as the cost of doing business."
Perhaps academia’s role is to be a testbed for potentially unprofitable ideas, or to create a place where profitable ones aren’t kept under wraps as trade secrets, Richmond suggests. And part of it is to act as a conscience. He doesn’t believe that ICT’s military funding presents a major ethical dilemma, because it’s rarely focused on actual combat; as he puts it, "the Army is one of the last areas that actually has decent research budgets to look at hard problems." But it’s also ICT’s job to scrutinize even the most benign technology — including virtual reality.
If the medium that ICT has spent decades working in goes mainstream, how will we use it? It’s a question that no one can answer. "We’re really struggling to figure out augmented and virtual and mixed reality, and what those spaces mean and how we’re going to have interactions that are meaningful," says Richmond. "And the only way to do it is by doing it. "