Menu

I shot guns with my eyes in virtual reality

The weird, clunky Fove eye-tracking headset is pure science fiction

  • on
5

At the Game Developers Conference, most meetings are held in utilitarian conference rooms or makeshift booths on the show floor, all inside the sprawling, maze-like Moscone Center. The VR startup Fove, though, is treating its demos more like an art show — in fact, it’s set up shop in a small San Francisco gallery. Besides the large screens playing a promotional video loop, an actual headset has been set up under glass, like a sculpture or historical curiosity. It’s a strange combination of dorky and chic, a fundamentally huge and clunky object cut into a clean, glossy prototype. But the really important part is hidden inside the goggles’ eyepieces: a pair of infrared cameras that add eye tracking to virtual reality.

Like its hardware, Fove — a San Francisco- and Tokyo-based startup whose name rhymes with “drove” — blends mainstream appeal with hardcore geekiness. Its CEO and co-founder, Yuka Kojima, says she came up with the idea for an eye-tracking headset while working on games at Sony Computer Entertainment. “I wanted to make interactive cinema to communicate with virtual characters using facial recognition, eye tracking. It was impossible,” she tells me. She started working on her own hardware part-time two years ago, then left Sony to found Fove last year.

Fove

Tobii, perhaps the best-known eye-tracking company, sells a wide range of gaming and professional hardware. Where it’s focused on 2D screens, though, Kojima thinks virtual reality is a natural fit. Earlier this year, Fove was in the news for "Eye Play the Piano," an eye-controlled keyboard for children with disabilities. At GDC, Kojima also talked about a system that would let paralyzed hospital patients communicate via eye tracking and a humanoid robot. "I think eye tracking will be the next user interface," Kojima says. "I have confidence about it."

But like Oculus founder Palmer Luckey, Kojima thinks games are the place to start, particularly because PC gamers are some of the only people buying consumer hardware powerful enough to handle high-end VR. While she thinks VR cinema is important, she worries that the current batch of experiences is too passive to keep people coming back. "The first time, everyone will be surprised, when [they] first wear the head-mounted display. 'Wow, amazing.' But after 15 minutes, they will stop playing and forget, and never return." Her solution, obviously, is eye tracking.

My experience with the Fove is a little bumpy — when I put the headset on, the computer decides that my eyelashes are my pupils. "We didn't expect that mascara would be black in infrared light," Kojima explains later; it’s apparently a problem they’ve been working to get around. After a little recalibration, it’s back on track, and I’m standing guard amid a cluster of cyberpunk skyscrapers, trying to keep myself safe from endless waves of flying robots. A lot of VR games, like the promising EVE: Valkyrie, ask players to shoot by aiming with their heads and using an Xbox controller to fire. Here, I’m aiming with my eyes instead.

It’s an unnatural feeling. I keep trying to turn and point directly at each enemy, but they’re coming too fast. My eyes, meanwhile, keep flitting around the screen, looking for new threats instead of focusing on my targets. Valkyrie effectively asks you to pretend there’s a gun strapped to your headset, but at least there’s some kind of physical action involved. After several misses, I start consciously keeping my head as still as possible, trying to remember that the objective isn’t really shooting drones so much as glaring them to death. It’s still difficult to turn and look in different directions, but I start realizing how useful my peripheral vision can be, and that my killing stares are surprisingly accurate. The demo ends just when I feel like I’ve started to figure it out.

Fove

The shooting gallery is flashy, but it’s a relatively simple use of Fove’s eye tracking. The team’s ultimate goal is to turn eye tracking into a common language for VR, whether that’s using it as the virtual equivalent of a mouse, making cinematic characters that know when the viewer is looking at them, or helping virtual cameras mimic the way that our eyes focus on objects in real space. The company is launching a crowdfunding effort in May, with hopes of releasing an SDK and headset next year.

Fove’s headset is bigger, heavier, and in several ways less technically sophisticated than its bigger competitors. Though Kojima comes from Sony, her work is unrelated to its Project Morpheus headset. ("I totally didn't know about Morpheus!" she says. "It was a secret project.") Beyond releasing the SDK, the company doesn’t have a firm plan for recruiting developers to work with a fairly niche interface. And we’re a lot more used to controlling things with our hands than with our eyes; there’s a reason Tobii doesn’t try to outright replace our mice and keyboards.

But Fove is part of a wave of companies that are trying to go beyond the simple VR headset. Virtual reality is increasingly about recording and responding to our physical bodies. It’s hard to imagine eye tracking becoming the primary way we interact with VR experiences, but it could end up being vital to interpersonal communication. Forget virtual characters, what happens when you’re hanging out with a friend online and want to make eye contact? Along with hand-tracking, controllers, and specialized hardware, it’s one potentially useful piece of a much larger puzzle — whether or not Fove is the one to provide it.

Correction March 25th, 4:50PM ET: Fove's crowdfunding campaign will launch in May, not today as previously stated.