When Facebook’s Mark Zuckerberg took the stage last year to show off a Facebook social VR platform, he expressed himself with an exaggerated set of emotions, created by different voice and body language cues. Compared to sophisticated non-VR face-capture cameras, this was primitive, but there was a good reason for that. Even though emotion is an incredibly important part of virtual presence, it’s tough to read somebody’s expressions when they’re hidden by a VR headset. But neuroscience and computing company MindMaze thinks it has a cheap and easy way to fix that.
The new MindMaze Mask, which was announced today, is a ring of electrodes that can be installed in any VR headset’s foam face mask. When you put the headset on, the mask detects which sensors your skin is touching with a certain pressure, then matches the pattern to one of 10 facial expressions, which an avatar reproduces. There’s also some software-based prediction, which is supposed to reduce lag between your face and your avatar’s. It’s not detecting and mirroring every movement — it’s more like you’re pressing button combinations with your face to call up animations. The example avatars are simple cartoons, but they can look like anything a developer wants.
The idea here isn’t to make a consumer product, but to have manufacturers build these sensors into headsets. MindMaze CEO Tej Tadi says that the company has been talking to several partners, and that headsets ought to be integrating them by the end of the year. That could include anything from niche products like the OSVR HDK2 to any of Microsoft’s new Windows Mixed Reality headsets, or even (although it seems less likely) the Oculus Rift, HTC Vive, or Gear VR.
Avatar expressions are still pretty limited
I got a short session in an OSVR headset studded with MindMaze Mask sensors. (I also saw the sensors embedded in an HTC Vive, and it’s supposedly also working on a Zeiss One mobile headset.) While I can’t speak to how well it works over the long term, it seemed to pick up some of my emotions with no calibration. I could smile and raise my eyebrows, and if I wildly exaggerated furrowing them, it guessed that I was angry. Winking didn’t work quite as well, but I could do it after a few tries. The system even caught when I was blinking, reflecting it with a split-second delay.
There are alternatives to Mask, but they’re either very limited or haven’t caught on yet. Eye tracking inside a headset can pick up some facial motion, but it’s a specialized and uncommon technology in VR headsets. Some avatar systems let you manually set an emotion with no special hardware, but this adds an extra step and doesn’t feel organic. Facebook doesn’t require custom technology or button-pushing, but it’s broadly extrapolating from larger head, voice, and hand movements.
At the same time, Mask is still effectively in the development kit stage, and it’s not perfect either. Specific elements, like the blinks, feel as though they’re reflecting me. But in most cases, I’m still telegraphing vaudevillian expressions to get emotions I don’t much care about. I don’t remember the last time I winked or bared my teeth at someone in real life, and the avatar doesn’t offer, say, an eyebrow furrow that’s perplexed rather than angry — or the blank-faced stare of death that I prefer for actual anger. Developers will be able to mix their own emotions, but their subtlety is limited to what the ring of sensors can pick up.
That doesn’t mean Mask is bad, though, just that I’m still figuring out how it might function in the real world. If the system is cheap (for now, Tadi says it adds $20 or $30), it could be worth using just to get a few simple and important emotions like smiling or surprise, which you’d otherwise need something like an emoji to express. On the other hand, even simple hardware like Mask could be overkill for things like blinking, which some avatars do already at automatic intervals. I might know it’s not a perfect match, but I doubt anyone else cares.
For now, the MindMaze Mask falls into the all-too-rare category of a VR easy win. It may not be a life-changing piece of sci-fi technology, but it approaches an obvious problem with a simple, clever, and competent solution.