clock menu more-arrow no yes

Filed under:

MindMaze's hand-tracking, mind-reading virtual reality headset is just as complicated as it sounds

New, 3 comments

Paste on the 'trodes

Swiss neurotechnology company MindMaze has conceived the ultimate — or at least somewhere around the penultimate — cyberpunk VR setup. Imagine a blocky virtual reality headset that looks like a bezeled Oculus Rift. Where a Rift developer might track your hands by mounting a Leap Motion to the front of the headset, MindMaze has integrated a Kinect-like combination of cameras and depth sensors, capable of finding your hands with some degree of accuracy. Instead of a strap, though, there's a rope-and-plastic net that covers the crown of your head.

And, theoretically, reads your thoughts.

The platform, called MindLeap, fuses a series of technologies that are increasingly being used for gaming: VR, motion capture, and electroencephalographic (or EEG) scans. EEG sensors can let users control everything from prosthetic limbs to plush cat ears, reading the brain activity that corresponds to general states of mind (like relaxation) or specific movements. MindMaze, originally a spinoff of the Swiss Federal Institute of Technology, uses this combination to help rehabilitate patients after a stroke, essentially trying to trick their brains into re-learning how to move paralyzed limbs. Here, at the Game Developers Conference in San Francisco, MindMaze has turned them into a short but complicated look at things to come in the world of entertainment.

MindLeap

To use MindLeap, a user puts on the display, cementing the net and some free-hanging wires to their head with conductive gel. From there, they'll enter a combination of virtual reality and augmented reality. Look to the left, and there's a simple virtual landscape; turn to the right, and the headset switches to showing the view from one of its front-mounted cameras, similar to Samsung's Gear VR. In augmented reality, its tracking sensors pick up the outline of a user's hand and set fire dancing around the fingertips. If the user is sufficiently relaxed, that fire turns to ice.

Enter the virtual reality labyrinth

A lot of work goes into making this little system function. While I could watch a demonstration of the EEG in action, MindMaze was reluctant to swab me with gel, so I only got the experience of an ordinary headset. The company readily admits it's no Oculus, and the prototype headset is solidly made but awkwardly front-heavy. It's also no game development studio, so the experience was a small, almost purely visual tech demo. Company CEO Tej Tadi says that he's hoping to sell a version of the "neurogoggles" to consumers, but not immediately; first, MindMaze is looking for partners and will be releasing a software development kit that works with existing EEG and headset technology in the "coming months."

MindLeap

The current web of mesh and wires may also be overkill for video games, says Tadi. In concept renderings for a final design, it's replaced with a simpler strip of gel-free forehead sensors, which would fit around the headset's faceplate. Fortunately, MindMaze was separately showing off a very similar band, which I was able to check out. The concept of brain-reading toys and gadgets isn't a new one — there's an annual conference dedicated to neurogaming — and the standalone band supported a brain-powered competition with only two moves. In it, each player is represented by a glowing sphere, trying to push a smaller ball into the opponent. To build up power and defend against attacks, you relax, letting your mind go blank (which, jet-lagged at GDC, turns out to be surprisingly easy). To attack, you frown, furrowing your brow cartoonishly and staring hard. For a sphere-themed fighting game played with relaxation and anger, it works pretty well.

MindLeap's designers talked about programming the headset to read the brain activity that corresponds to certain motions, or even more abstract emotions like happiness; as long as there's an identifiable and repeatable pattern, it's possible to map a control system onto it. Paired with motion capture, MindMaze suggests that this "thought-driven gameplay" could solve VR's controller program, but that's still a long ways away. Mind-controlled prosthetics fill an unavoidable need, but in gaming, we're a lot more used to opening a virtual door by pressing a physical button than we are turning an imaginary handle.

What this does represent is another foray into creating holistic VR systems. As Oculus drags out the development of its Rift head-mounted display, and companies like Samsung and Valve dip their toes into the water, we're seeing a proliferation of weird control systems to match — among other virtual reality devices at GDC, we'll be taking a look at a headset that's also an eye-tracker. MindLeap is far more concept than product, but since nobody else has VR headsets figured out either, why not throw on some electrodes and see what happens?