Skip to main content

Cracking Kinect: the Xbox One's new sensor could be a hardware hacker's dream

Cracking Kinect: the Xbox One's new sensor could be a hardware hacker's dream


Will a higher-resolution camera lead to more inventive DIY creations?

Share this story

Kinect Xbox One 560
Kinect Xbox One 560

One of the biggest upgrades to Microsoft's recently unveiled Xbox One console is the new and improved Kinect: the device now features a higher fidelity sensor, a larger field of view compared to the original, and better skeletal tracking. This could have some potentially cool applications when it comes to watching TV on your console or playing games, but, like the original Kinect, the most exciting new ideas will likely come from outside of Microsoft and traditional game developers. The hackability of the original Kinect created a healthy DIY movement, and while it's too early to tell if the new version will do the same, there are definitely plenty of possibilities — especially with the recently announced Kinect for Windows launching sometime next year. But will the new Kinect be as open to developers as the original?

Will the new Kinect be as open to developers as the original?

Much of the possibility stems from the higher-resolution camera built into the new Kinect. It utilizes a Microsoft technology dubbed "Time of Flight," which, according to the company, can measure "the time it takes individual photons to rebound off an object or person to create unprecedented accuracy and precision." In our time with the device, it could tell not only that you were moving a thumb, but which way that thumb was facing.

"The ability to measure minute movements of fingers / toes makes for some really nice possibilities for interaction," says new media artist and game designer Matt Parker. And according to Oliver Kreylos, a virtual reality researcher at the University of California, Davis, "it might just be enough to push the Kinect 2 into reliable 3D finger detection, which would open up a large new application area. 3D finger and head tracking are the most important things for me, and Kinect 1 just didn't support them well enough." On his blog, Kreylos calls the new tech "kind of a big deal."

According to music producer and Kinect hacker Chris Vik, the ability to detect individual finger movement could solve one of his biggest problems with the original sensor. "The main issue I have with using motion capture as a controller at the moment is that there are no buttons; there's no way to tell the system that you now intend to control something. There's no trigger," he tells The Verge. "If the new Kinect can successfully detect touching your index finger to your thumb, then gestural control will be much less flaky and awkward to use. This is the reason touchscreen interaction works so well, because you choose when you touch the screen to control it." One of Vik's more recent Kinect projects involved using the device to control an 83-year-old, four-story-tall organ in Melbourne.

"It might just be enough to push the Kinect 2 into reliable 3D finger detection."

In addition to opening up new ideas for creators, it should also help improve existing projects. Ubi Interactive is a startup in Munich that's developing a Kinect-based virtual touchscreen using a projector. Though it's still in development, early reports have praised its responsiveness, and Microsoft even awarded Ubi $20,000 as part of its Kinect Accelerator program. With the improved sensor, it could be even better. "Though we do not know the exact details on the resolution and added precision of the depth-image stream," says Ubi co-founder and CEO Anup Chathoth, "any improvement in this regard will help us a lot in improving the precision." He also cites features like decreased latency and the wider angle of vision as likely improvements.

Aside from the increased precision, Kinect hackers could also benefit from the new active-infrared features in the device, which Microsoft claims can work in "nearly any" lighting condition. "This will offer developers better built-in recognition capabilities in different real-world settings," the company claims. If it works as advertised, this could potentially bring Kinect creations into new spaces. "One huge limit in designing with Kinect was the inability to have games / experiences played in sunlight, and a number of other issues being in situations where the lights throw enough IR light to confuse the Kinect," says Parker, whose work includes a number of large-scale installation pieces.


The big question is whether or not the new Kinect will be as open a platform as the old. Unlike the original, the new Kinect won't be sold separately initially, instead being bundled with each and every Xbox One console. For now, Microsoft is keeping quiet about the new device's DIY potential, though details about the software development kit are expected next month at the company's BUILD 2013 event. "We'll share details about how developers and designers can begin to prepare to adopt these new technologies so that their apps and experiences are ready for general availability next year," wrote Bob Heddle, director of Kinect for Windows. We reached out to Microsoft about its plans but have yet to hear back as of this time.

"This is going to be a better tool for experimentation and creative applications."

In the meantime, developers are excited about the prospects, and cautiously optimistic that Microsoft will be as open as it was last time around. "There's no question that more advanced technology like this is going to be a better tool for experimentation and creative applications," says Vik. "The question is just how Microsoft intend to approach the political side of things after they were slow to react to people ‘hacking' the original Kinect."

Today’s Storystream

Feed refreshed An hour ago Striking out

Andrew WebsterAn hour ago
Look at this Thing.

At its Tudum event today, Netflix showed off a new clip from the Tim Burton series Wednesday, which focused on a very important character: the sentient hand known as Thing. The full series starts streaming on November 23rd.

The Verge
Andrew WebsterTwo hours ago
Get ready for some Netflix news.

At 1PM ET today Netflix is streaming its second annual Tudum event, where you can expect to hear news about and see trailers from its biggest franchises, including The Witcher and Bridgerton. I’ll be covering the event live alongside my colleague Charles Pulliam-Moore, and you can also watch along at the link below. There will be lots of expected names during the stream, but I have my fingers crossed for a new season of Hemlock Grove.

Jay PetersSep 23
Twitch’s creators SVP is leaving the company.

Constance Knight, Twitch’s senior vice president of global creators, is leaving for a new opportunity, according to Bloomberg’s Cecilia D’Anastasio. Knight shared her departure with staff on the same day Twitch announced impending cuts to how much its biggest streamers will earn from subscriptions.

Tom WarrenSep 23
Has the Windows 11 2022 Update made your gaming PC stutter?

Nvidia GPU owners have been complaining of stuttering and poor frame rates with the latest Windows 11 update, but thankfully there’s a fix. Nvidia has identified an issue with its GeForce Experience overlay and the Windows 11 2022 Update (22H2). A fix is available in beta from Nvidia’s website.

External Link
If you’re using crash detection on the iPhone 14, invest in a really good phone mount.

Motorcycle owner Douglas Sonders has a cautionary tale in Jalopnik today about the iPhone 14’s new crash detection feature. He was riding his LiveWire One motorcycle down the West Side Highway at about 60 mph when he hit a bump, causing his iPhone 14 Pro Max to fly off its handlebar mount. Soon after, his girlfriend and parents received text messages that he had been in a horrible accident, causing several hours of panic. The phone even called the police, all because it fell off the handlebars. All thanks to crash detection.

Riding a motorcycle is very dangerous, and the last thing anyone needs is to think their loved one was in a horrible crash when they weren’t. This is obviously an edge case, but it makes me wonder what other sort of false positives we see as more phones adopt this technology.

External Link
Ford is running out of its own Blue Oval badges.

Running out of semiconductors is one thing, but running out of your own iconic nameplates is just downright brutal. The Wall Street Journal reports badge and nameplate shortages are impacting the automaker's popular F-series pickup lineup, delaying deliveries and causing general chaos.

Some executives are even proposing a 3D printing workaround, but they didn’t feel like the substitutes would clear the bar. All in all, it's been a dreadful summer of supply chain setbacks for Ford, leading the company to reorganize its org chart to bring some sort of relief.

External Link
Jay PetersSep 23
Doing more with less (extravagant holiday parties).

Sundar Pichai addressed employees’ questions about Google’s spending changes at an all-hands this week, according to CNBC.

“Maybe you were planning on hiring six more people but maybe you are going to have to do with four and how are you going to make that happen?” Pichai sent a memo to workers in July about a hiring slowdown.

In the all-hands, Google’s head of finance also asked staff to try not to go “over the top” for holiday parties.

External Link
Insiders made the most money off of Helium’s “People’s Network.”

Remember Helium, which was touted by The New York Times in an article entitled “Maybe There’s a Use for Crypto After All?” Not only was the company misleading people about who used it — Salesforce and Lime weren’t using it, despite what Helium said on its site — Helium disproportionately enriched insiders, Forbes reports.