Skip to main content

The all-seeing Kinect: tracking my face, arms, body, and heart on the Xbox One

The all-seeing Kinect: tracking my face, arms, body, and heart on the Xbox One

Share this story

Kinect for Xbox One David looks dumb
Kinect for Xbox One David looks dumb

"It's what's on the inside that counts" is something of an unofficial mantra of Microsoft's new Xbox One. The Kinect is perhaps the best example: externally, it's more or less unchanged from its predecessor, but it's actually a very different device. It's been upgraded in a huge way, but the end result is simple: the Kinect just sees more. (And hears more, but that's another matter.)

It has an ultra-wide 1080p camera (which should mean the Xbox One doesn't require such a large room), which easily picked up all of the dozen or so people sitting in the "living room" testing lab on Microsoft's campus. Kinect can even see in the dark, thanks to an infrared sensor that engages when the primary camera can't see anything. Along with higher-end processing power and a host of new software, Kinect feels a bit like it's gone from usable prototype to real, legitimate product.

Kinect has always been able to tell that you're moving. But now it can tell if you're moving your thumb, and which way your thumb is facing. It can tell which muscles you're engaging at any given time, and how much — it knows the difference between a jab and an uppercut, and registers them differently. If you're playing with a friend, it can tell when the two of you switch places, or even when the two of you switch controllers. Kinect knows if you're smiling or frowning, or if you're talking or not. It knows if you're looking at the screen or not, and will only register your commands if you're looking. It knows, by either remarkable science or sorcery, your heart rate just by looking at your face.


Kinect knows when you're talking, flexing, or smiling

We spent a few minutes in a crowded room using a prototype of the new Kinect, and we left reeling. There's almost no latency, things are astonishingly accurate — the muscle sensors knew even the slightest shift in my posture, and try as I might I couldn't make it think I was smiling when I wasn't. We heard it discern commands from a noise-filled room, and track our movements in the dark when we couldn't see them ourselves. It was a fairly controlled situation, though, and we're curious to see how it holds up in the real world.

The real world applications are really the whole game here. Kinect's raw capability is absolutely remarkable, but how developers will build them into games remains to be seen. Could a boxing game know the difference in how hard I punch? Could it know how tired I am based on my heart rate, and knock me out more easily? Could a shooter recognize that I'm too relaxed, and ratchet up the intensity to get me back in?

More than anything, that's what we're excited to see about Kinect and the Xbox One: how the immense amount of data turns into a more fun, more immersive gameplay experience. But at least at first blush, the data part seems to have been pretty much solved.

Xbox One Kinect hands on photos