Skip to main content

The Oculus Rift meat puppets are here

The Oculus Rift meat puppets are here

/

How would it feel to control a real person in virtual reality? The Pretender Project is a rough sketch

Share this story

If you buy something from a Verge link, Vox Media may earn a commission. See our ethics statement.

Body swapping is a dream of both science fiction and present-day virtual reality. What would it be like to spend a day as a model, or a rock star, or a child? Designer Yifei Chai wants his Pretender Project to answer that very question. But in the process, he's learning how hard it can be to condense a collective cultural dream into a real project.

Visit the Royal College of Art to see Chai's project, and you might be asked to don a pair of Oculus Rift virtual reality goggles and stand in front of a Kinect. Suddenly, you'll be seeing out of someone else's eyes — the eyes of your "avatar," probably Chai. Put your hand up, and an electric current will lift his too. Bend your elbow, and his will follow, clumsily. It's not just imitation. It's a reflex, like a dead frog's legs twitching when you shock it. "The tingling as my arms are moved and twisted into positions out of my control is more powerful than I imagined," is how New Scientist's Sandrine Ceurstemont describes it. "It is quite unsettling."

"I was interested in what would happen if you put a living person in another living person."

The Pretender Project started as a way to build on Chai's interest in, among other things, witchcraft. Fascinated by the idea of transferring your mind into another body, he soon turned his attention to the Proteus Effect: the phenomenon of changing real behavior to match a virtual avatar. "I was really interested in what would happen if you put a living person in another living person," he says. To do so, he hacked together a set of three common technologies. One person, who he calls the "experiencer" or the "controller," wears an immersive virtual reality headset like the Rift. Their motion is tracked by the Kinect and translated into a 40- to 80-volt shock, which targets specific muscles in the avatar — it's essentially, says Chai, the same process would-be strongmen use to tone their body without working out. The overall effect is a very rough sketch of something from Neuromancer, or a lo-fi James Cameron movie.

In theory, the avatar could be directed to do anything. A project video shows them clubbing, eating breakfast, riding a bike, and escaping a gunfight. But the real system can only control your arms, and it does so with high lag and low accuracy. Chai says a shock can give you a wide range of motion, but it can't target small muscle groups. His current-generation Kinect, he admits, is "very, very poor" at skeleton tracking. If you cross your arms, it could well lose track of where they are. A higher-definition Oculus Rift and next-gen Kinect for Windows are both in the cards, but there are basic, fundamental problems preventing people from living in another body. The only way to get truly fine muscle control "would be for the pinpointers to be directly implanted under the skin," says Chai. Legs, which bear the load of the body, would be difficult to shock into motion at all. And touch feedback isn't even part of the equation.

The real system controls your arms with relatively low accuracy

Chai originally planned the project simply as a way to explore a concept. It built on The Machine to be Another, which allows two people to swap perspectives and move in tandem, experiencing the sensation of being in a different body. "A lot of the inspiration for the project came from them, but at the same time, I pushed it to a new level," he says. While the motion in The Machine to be Another was choreographed by a human, the Pretender Project's is automatic. Both have been described as a way to build empathy by putting a wearer almost literally in someone else's shoes. Leaving aside the idea of glamorous virtual vacations, there's the opportunity to simply see how the world looks from the perspective of a different body, as long as that body is all right with being temporarily turned into a puppet.

Things have gone further than Chai expected. "To be honest, I hadn't expected this project to get so much attention," he says. He's interested in continuing it, but among other things, he has little training in coding. "I would need a team of people with more expertise than mine to push this forward. But I'm definitely willing to see what I can get out of this." Compared to other parts of the academic world, Chai's work is low-tech. At the University of Washington, you can actually control someone's body with your mind. Despite this, it's captured our attention because it feels like fiction. No matter how well it works — and, by all accounts, there are some very hard limits in play — it's a testament to how even the technology we associate with toys and spurious infomercials can do something that seems to belong in the world of mad science. After all, science fiction isn't just about predicting the future. It's about pointing out that the present is already deeply weird.