Skip to main content

The race to control video games using your mind

The race to control video games using your mind


Your next console probably won't have a brain-based controller, but the one after that might

Share this story

Virtual reality and the ability to control things with our minds are two flights of fancy that have long enchanted scientists and sci-fi geeks alike. The first is rapidly heading toward becoming a reality thanks to the explosive interest in the Oculus Rift, but the second is in active development. The NeuroGaming Conference and Expo that took place in San Francisco this week offered a good opportunity to see what the next frontier of gaming might look like after VR takes hold. For now, though, it’s a trend very much in its infancy.

That’s not necessarily a bad thing. I arrived expecting an exhibition filled with half-baked game ideas. But the event was more focused on how games can help the mind than games you play *with* your mind. Akili Interactive, for example, built a seemingly standard iPad game that measures the differences in brain activity between a normal child and one with ADHD or autism. The company says it measures 65 different pieces of data every second through gameplay. "We have very clear signatures that come out of the game that tell us exactly what a person’s cognitive deficits are," says Akili’s creative director, Matthew Omernick.

A company called Qneuro showed off a game that pulls together all the concepts the NeuroGaming Expo focused on. It’s an educational game called Axon Infinity: Space Academy with a futuristic, outer space style in which you learn math skills and put them into play in missions when fighting aliens. The catch here is that if you use an EEG, the game can adapt and gets more difficult based on brain readings. The math game can still be played on an iPad, the Surface, or a standard PC with a standard controller, but it will change based on both your aptitude and brain response.

More of a research convention for scientists than your average gaming conference

The event was full of representatives from universities and research groups like the University of Texas’ Brain Performance Institute, the nonprofit Center for Responsible Brainwave Technologies (a group trying to regulate how brain data is collected and used by companies), the Cerebral Palsy Network Program, and a number of others. It felt more like a research convention for scientists to talk shop than any sort of gaming expo I’ve ever seen.

Video games to sharpen your brain, not necessarily controlled by your brain

Adam Gazzaley, director of the Neuroscape lab at UCSF, set the tone with his opening keynote. One of the main studies he discussed involved a custom video game built to increase cognitive factors (like attention span, multitasking ability, memory, and so one) in older adults. The game is a racing game called Neuroracer that’s designed to engage the multitasking portion of your brain. Results showed that 12 hours of playing the game over a month significantly increased the cognitive function of a test group of 60- to 85-year-olds compared to a control group, Gazzaley said.

What struck me about the NeuroGaming event was how little "gaming" there was to be had. Last year, a game called Throw Trucks With Your Mind delivered exactly what it promised. Players wore an EEG reading sensor and when they slowed their thoughts and blocked out distractions, their video game avatar was able to start picking up and throwing objects at their opponents. It sounded like a fun demo, but unfortunately things don’t seem to have progressed much from that idea.

There was one game called NeuroMage that used a similar "relax the mind" technique to help you learn spells you could then cast against an opponent, and a Star Wars tie-in from Uncle Milton’s Toys used the same relaxation technique to let you levitate a hologram of the Millennium Falcon. The Star Wars product will actually come to market this fall — it works with a tablet to project holograms and will come with a custom EEG headset from NeuroSky to measure brain activity. (There truly is nothing that Disney won’t consider as a Star Wars tie-in.) What you’ll be able to do with it beyond levitating a spaceship hasn’t been revealed yet.

Still, you’d be disappointed if you were looking for the future of gaming here. Indeed, most of the gaming-related demos being shown off weren’t directly tied into neuroscience at all. There were plenty of demos using the Oculus Rift, Sony was showing off Project Morpheus, and Leap Motion was on display. The Sixense lightsaber demo we saw at CES attracted quite a crowd. With these demos, the actual connection to neuroscience seemed to be limited to the idea that motion control and VR could be integral parts of brain-based gaming — but the feeling I got was that they were just there to be part of an expo that has its roots in nontraditional gaming controls.

Neurogame builders have been encouraged by the VR boom

The belief I got from talking to various exhibitors was simply that mind control can be another tool that adds to the experience of gaming in a virtual world. We have the Rift for VR, Sixense or Leap Motion for tracking our movements, and companies like NeuroSky and Emotiv providing brain-sensing technology. The pieces haven’t come together yet, but there’s a hope that they will someday soon. (MindMaze, which we saw at GDC 2015 is a good example of something trying to tie all these bits together.) While your brain isn’t a sure bet to be the next radical input method for next-generation consoles, all of the tinkering and experimentation happening around neurogaming suggests it will be eventually.