Meta’s sci-fi haptic glove prototype lets you feel VR objects using air pockets

Meta Reality Labs

You cannot pet a dog in Meta’s new, high-tech virtual reality gloves. But researchers are getting closer.

Meta (formerly Facebook) is known for its high-profile moves into virtual and augmented reality. For seven years, though, it’s been quietly working on one of its most ambitious projects yet: a haptic glove that reproduces sensations like grasping an object or running your hand along a surface. While Meta’s not letting the glove out of its Reality Labs research division, the company is showing it off for the first time today, and it sees the device — alongside other wearable tech — as the future of VR and AR interaction.

At a simplified level, Meta’s haptics prototype is a glove lined with around 15 ridged and inflatable plastic pads known as actuators. The pads are arranged to fit along the wearer’s palm, the underside of their fingers, and their fingertips. The glove also acts as a VR controller. The back features small white markers that let cameras track how the fingers move through space, and it’s got internal sensors that capture how the wearer’s fingers are bending.

When you put on the glove and enter a VR or AR experience, a sophisticated control system adjusts the level of inflation, creating pressure on different parts of your hand. If you’re touching a virtual object with your fingertips, you’ll feel the sensation of that object pressing into your skin. If you’re gripping a virtual item, the long finger actuators will stiffen, creating a sensation of resistance. These sensations work alongside visual and audio cues to produce the illusion of physical touch.

The tech draws on the relatively new field of soft robotics, replacing bulky motors with tiny air valves. Meta has been working on it nearly since it acquired the Oculus VR startup in 2014. It developed its first prototype — one finger with a single actuator — in 2015.

One of the first experiences that Reality Labs head Michael Abrash recalls was looking at a virtual plate from inside a VR headset — where a single actuator, combined with the virtual image and the sound of rubbing the rough ceramic, was incredibly convincing. “I saw the plate, and I saw my finger on the plate, and I heard the sound — that kind of scraping sound across it — and I felt the vibration,” he says. “And I will tell you, I was running my finger over a ceramic plate.”

Simulated touch isn’t an entirely new phenomenon. Even combining VR with something as simple as controller vibrations can make people feel more like they’re touching something, and many companies have worked on wearables that either track users’ hands or provide haptic sensation. Some even integrate temperature sensations, which Reality Labs isn’t prioritizing.

But Meta is poised to mass-market haptic gloves in a way other companies can’t. Most haptics device makers sell specialized products for military, industrial, or academic institutions. By contrast, Meta produces the dominant consumer-oriented Quest VR system and is putting billions of dollars toward building a “metaverse” that integrates VR and AR. If it launches a haptic glove system, Meta can guarantee the system will work on the Quest and encourage app developers to use it.

Reality Labs imagines gloves being one of multiple controller methods for future glasses and headsets, alongside more lightweight solutions built on electromyography or EMG — a system that reads nerve signals on your arm and translates them into digital input. Meta acquired EMG wristband company CTRL-Labs in 2019, and the EMG team works separately from the haptic design team, although the two technologies could easily overlap.

“[Doug] Engelbart and Xerox PARC are the only time that fundamentally the way we interact with the digital world has ever changed,” Abrash says — referring to the invention of the mouse and the Alto desktop computer design in the 1970s, something that helped set the course of modern personal computing. “AR glasses are going to require that to happen. You’re not going to walk around with a keyboard and mouse. You’re not going to be taking out your phone to interact with it.” For some tasks, you might want a simple wristband. For others, you’d want fuller tactile sensations.

There are still many barriers to producing consumer-ready haptic gloves, though.

On one hand, the team wants to dramatically increase glove actuator density, going from tens to hundreds or even thousands within a few years. Right now, the gloves can offer a sense of objects’ contours, but not fine distinctions between surfaces — it relies instead on the suggestive power of audio and images. “You could pet a dog, but you wouldn’t feel the texture,” says Reality Labs engineer Katherine Healy. “You need high-density actuation to be able to really get that sensation, and this glove does not do that.” Meta might not be prioritizing something as specific as fur, but it wants to produce more broadly high-quality sensation.

On the other hand — or more literally, the same hand — the gloves need a dramatic size reduction. Meta’s prototype is svelter than something like a ‘90s Nintendo Power Glove, but that’s not saying much. Healy says the device needs to be light enough that people feel like they can normally interact with the real world as well as the virtual one, and it needs to go fully wireless, rather than relying on a tether like it does now.

To make things even more complicated, haptic gloves need to fit precisely against wearers’ skin. That could require designing differently fitting versions for each buyer, possibly through a process like 3D knitting, which can produce an object of clothing from a custom-sized digital design. Then there are practical questions like how to clean a high-tech glove — right now, it’s carefully wiped down with alcohol. “With garments in general, we expect them to be washable,” says Healy. “We would love to be able to create a glove that could be washed. How? We don’t know yet. But that’s part of our vision.”

If the gloves are commercialized, they’ll face the same privacy and security challenges as other wearables that collect biometric information. Abrash compares using technology like EMG wristbands to typing on a keyboard, something that’s rarely described as a potential threat. But wearable technology can analyze people’s physical motion at a fine level, while even basic typing patterns can predict the onset of diseases like Parkinson’s. Meta will need to establish policies governing how much of that data leaves users’ devices and who else has access to it.

For now, the team is just trying to work out how real is real enough for VR touch. “One of the interesting things about haptic gloves is that obviously we can’t reproduce reality exactly, which is different from audiovisual stuff,” says Abrash. A super-dense screen could be almost distinguishable from a real image, and a sound wave could capture the sensation of hearing somebody speak. But Meta can’t realistically make a glove that will stop your hand from going through a virtual table. Instead, it has to find the point where you’ll suspend disbelief and accept that the table is there, even if it objectively feels more like jello than hard wood. “There’s a new physics where nothing is solid on a large scale.”

And as for the dog? “It seems likely to me that if the haptic gloves work out, you’ll be able to pet a dog, but it will be a virtual dog and it will be a slightly different experience,” says Abrash. “But basically — emotionally, experientially — it will feel as real.”

Comments

This looks really cool, but it’s just such a shame it has to come from Facebook.

Why? Getting a bit tired of the "fAcEbOoK sUcKs" meme. Everyone claims to be deleting Facebook and hating the company, yet almost half of earth’s population has an account.

At least they are trying something new and trying to push technology forward, instead of sitting around and releasing the same products every year with minor upgrades.

Honestly I don’t even care much for Facebook, but it’s getting tiring seeing the sentiment that Facebook is shameful or evil or something.

Yes, they’re trying something new and pushing tech forward, that’s why I said it looks really cool.

But its creation is funded by money made by violating people’s privacy and a whole load of other despicable actions, which is why it’s a shame.

There’s a simple way to get people to stop pointing out the may ways in which Facebook is a shitty company—they just need to stop being a shitty company.

It’s not a meme. This shit is real. FaceBook and its advertising model have done untold damage to ‘democratic’ nation states, has deliberately heightened ethnic tensions in order to increase engagement, and is generally a massively reprehensible piece of shit on the shoe of humanity.

Don’t trivialize this as a meme.

No worries, it’s coming from Meta instead!

… no? Not fooled? Dang.

"[Doug] Engelbart and Xerox PARC are the only time that fundamentally the way we interact with the digital world has ever changed,"

So we’re just going to ignore multi touch?

MOAD has a certain hero-worship level of fascination for late middle-aged white technologists. Multi-touch was developed at an industry level after the iPhone provided the proof of demand for all-screen and general touch-done-beautifully, with PCAP technologies being matured/productized in Asia.

Of course that kind of distributed success isn’t a very sexy story, and doesn’t fit within the "heroism of a single person" narrative such folks like to see.

Getting closer to sci-fi, and I am starting to get concerned.

I’m sure this will take off in experimental use cases for the medical field, specifically training and scenario tests. Then eventually, we’ll see it in the porn and sex toy industry and THAT’S when things will get…interesting.

THAT’S when things will get…interesting messy.

FTFY

So in the future, the only way to see your meta-friends will be wearing AR/VR goggles, and the only way to pet your meta-dog will be wearing a haptic glove. But you’ll never be able to kiss your meta-wife, and never hold your meta-children. Maybe we should stop trying to simulate everything, and just go outside, meet real people, and live real life!

Metadivorces are cheaper

Oof. Looks like their CTRL labs acquisition is being dumped! If they still have to resort to vision based hand tracking, I can only wonder where this will land them.

And BTW, for folks that think this is new stuff, please – versions of haptic gloves have been pushed around by hucksters every half a decade or so (look up data glove, Jaron Lanier’s nonsense, P5 glove etc). Newtonian mechanics and perceptual/proprioceptive thresholds are just generally a bitch.

I would like to remind everyone that the tech press, including very much this publication slobbered real hard over Magic Leap so don’t take any of this VR/AR/metaverse BS seriously.

View All Comments
Back to top ↑