Skip to main content

Microsoft uses Kinect to let you navigate Windows using air gestures

Microsoft uses Kinect to let you navigate Windows using air gestures

Share this story

Kinect WIndows gestures
Kinect WIndows gestures

Microsoft is revealing its latest research project today, a way to use Kinect for gesture-based navigation with Windows. While the company has long supported Kinect for Windows, it hasn't really provided a way for Windows users to navigate using Kinect. A new research project aims to solve that by mounting a Kinect camera above a desk that detects a number of gestures above and around a keyboard and mouse.

While it looks similar to the Leap Motion approach, it's not designed to replace the keyboard and mouse says Microsoft principle researcher Abigail Sellen. "What we don't want here is Tom Cruise in minority report," Sellen explains. "What gestures are good at are being very casual, expressive. What they're not good at is being precise. Touch is good at that, mouse pointer is good at that. We don't need to do that with gestures." Instead, Microsoft has built a way to let keyboard and mouse users take advantage of gestures to navigate. You can swipe to move backwards and forwards in a browser, flip your hand to tile Windows, and even make a search gesture to bring up the search Charm in Windows 8.

Microsoft's aim with the project is to explore how such gestures and interactions might enhance desktop computing, rather than replace it. "We're not doing anything precise, and nothing that requires you to hold your hands up for a long time," explains Sellen. Microsoft is also exploring interaction areas next to the keyboard where you could perform actions like scrolling without a dedicated piece of hardware. Those particular gestures are very experimental, but we got to try some of the pinching and zooming techniques and they work well. While they didn't always work first time, being able to flip over your hand to control an interface is very natural providing you remember all these gestures.

Sellen's team has been testing the system in a lab and is trying to determine what works well. "It taught us which gestures people find the most usable," explains Sellen, noting that there's a high bar to try and get people to change their normal practices. Like many other research projects at Microsoft, it's simply research, but a number of Microsoft's Kinect-related projects have made it into final form so there's hope that the company could bring this to life in future, smaller, versions of the Kinect.