As virtual reality gains steam, the question of virtual worlds is never far behind. Philip Rosedale is best known for online community Second Life. But since last year, we've been watching for news on High Fidelity, a new project meant to blend his previous work with cutting-edge telepresence technology. The system, announced in 2013, was compared to the OASIS of Ready Player One: a series of worlds connected to each other by a central network and economy, provided — obviously — by Rosedale himself. At the Silicon Valley Virtual Reality Conference in Mountain View, we're seeing the first hints of what that could mean, as well as a sense of the many hurdles left to jump.

Where Second Life is housed on hundreds of thousands of Linden Lab machines, High Fidelity worlds would be distributed across the user base. "We could potentially go from 600,000 or so servers to 600 million," says Rosedale. If you're on one world, you can see others in the sky and travel between them, watching blocky Minecraft-like environments resolve to detailed landscapes as you get closer. As computers advance, so will the possibilities. "The virtual worlds of the future are going to look like Pandora from Avatar. And once we can build things in those worlds, I would assert that even people who are passionate about VR, here, have no idea what is coming," he says. "We're going to be arguing over the price of real estate in the forests of Pandora."

Img_5886

The core of High Fidelity, however, wouldn't be graphics; it would be responsivity. Using sophisticated motion capture techniques, you could mirror your head movement and facial expressions onto an avatar. Using a full-body harness, you could go a step further, moving your arms and torso naturally to interact. All the while, you could be in an immersive environment that puts a Skype call to shame. As Rosedale delivered his speech, his avatar loomed overhead, speaking and nodding. His partner, wearing a harness, played air drums and delivered a hug. Second Life and reality, bleeding together.

When I pulled my hair back, my avatar brushed the side of his face

That was, in any case, the theory. But when you boil that grand vision down to a prototype, it's a rough one. The High Fidelity demo, first shown earlier this year, is just a crudely rendered dance floor — it's not trying for the verisimilitude of Second Life, let alone next-generation graphics, and it might not be able to do so for some time. When I connected, the AI club-goers were all dancing on the same square foot of ground like some eight-legged twerking monstrosity. More importantly, face-tracking remains an inexact science. Rosedale's avatar could look perfectly natural one moment and jerk mechanically the next. But when it did work, slipping on a Rift and seeing an avatar perform tiny, very human shifts of the head and mouth could be downright eerie.

The PrioVR harness High Fidelity uses for body tracking, to anyone who has only used crude sensors like the Kinect or PlayStation Move, is even more wonderfully uncanny. Strapped into wrist, elbow, and chest sensors, I held my arms up for calibration. When I dropped them, the movement was instantaneous and precise, the kind of connection I'd never had with a motion controller. I could wave and see the movement of my entire arm, stretch and see my elbows bend realistically. When I pulled my hair back, my avatar brushed the side of his face. I never got a hug, but the demo left me wanting more.

None of these parts, though, fit together. The High Fidelity demo didn't support using the Oculus Rift and PrioVR harness at the same time, so I had to pick between visual immersion and realistic movement. That's a solvable technical problem, but it makes it difficult to envision a finished product. A more intractable one is VR and face tracking. Right now, you simply can't let a camera read your face while wearing a box over most of it. The's a lot that High Fidelity could do, but how much of it will actually cohere into a finished product?

Rosedale expects some version of High Fidelity to launch in about a year, but he's made parts of the project open source, hoping that enthusiasts will help create and manage worlds. Second Life has survived because of a dedicated user base and steady economy, which kept going long after the hype had dissipated. It's harder to tell what High Fidelity can achieve, especially given how much experimental hardware it requires. But what it's promising, no matter how tentatively, is something online communications have been trying and failing to approximate for decades: a sense of real, physical connection.