I got my first look at the Leap Motion Controller in an outdoor tent adjacent to the South by Southwest convention center months before it arrived on my desk. I found an empty computer and played a quick round of a Jenga-like game called Block54, hovering my thumb and forefinger over a gum pack-sized device to pinch and remove pieces. I found it incredibly difficult.
I played a lot more Block54 this past weekend in anticipation of the Leap Motion Controller's official launch today. This time, I was in the privacy of my own home instead of being surrounded by gawking strangers eager for their turn at the machine. I grew significantly better than my initial attempts in Austin, Texas, but I still struggled the entire time. Finding the right depth and height took practice, but wasn't too much of a problem after a few rounds.
The main problem was the pinching. My fingers longed for the polished wood grain that would tell me I had achieved the proper width. I successfully completed round after round, but the task felt weird and disconnected.
"Do things on your computer just like you do them in real life."
Leap Motion is a touchless user interface, allowing users to interact with a computer by waving a finger or hand. It's in the same category as a keyboard, mouse, touch screen, or DualShock controller; it turns physical actions into data that can be recognized by a computer.
Leap Motion's grand vision is to let you "do things on your computer just like you do them in real life," but the controller doesn't fulfill this promise. When I select my floor on an elevator, I can feel the button pushing back. If I grab a baseball, I can position my fingers to throw a curveball by feeling the curved line of stitches accenting the smooth cowhide surface. With the Leap Motion Controller, my hands just float freely through the air.
There's no way of telling when something is touched or grabbed
The controller's initial setup is completely painless: after a quick software install, I was launched immediately into the Orientation app, which helps show exactly what the controller can see. I waved my hands over the device and watched little particles react to the gentle motion. Next, the app showed a digital mockup of each of my hands, matching my movements in real time. The entire process was extremely impressive, despite a few misread fingers here and there.
The proprietary Airspace Store has around 75 different apps at launch, a respectable amount for such a new technology. Some applications try to provide value, such as the $3.99 educational app Frog Dissection, but others are just weird. Within the Experimental category is a free app called Lotus, which has four different audio and visual environments that are manipulated through movement. I was amazed as I explored each stage in Lotus, but my enjoyment lasted no more than ten minutes and I'd only open the app again to show a friend.
As I was struggling to pin down a frog's legs in preparation for my digital scalpel, I couldn't help but think of Bret Victor's A Brief Rant on the Future of Interactive Design. By his definition, the Leap Motion Controller fails as a tool because it does not "[address] human needs by amplifying human capabilities." It ignores our hands' ability to both feel and manipulate. It's a step behind the touchscreen, despite being a newer technology, because there's no way of telling when something is touched or grabbed.
Our hands are not very precise when floating through space
The Kinect is a touchless interface that's slowly incorporating a more natural type of interaction. Players may be able to call reinforcement with an arm wave or remain completely still to avoid detection. Holding and aiming an invisible gun feels unnatural, but players may already be unconsciously ducking to the side when avoiding an attack.
The problem isn't that Leap Motion lacks precision - it can track 10 fingers to within 1/100th of a millimeter. It's that our hands are not very precise when floating through space. Apps that don't rely on specific spatial placement provide the best experiences. My favorite app is Double Fine's Dropchord, a $2.99 platformer where players must collect music notes with a beam that runs between two fingers. The game doesn't let go once a finger is associated with a side of the beam. Moving too much, or accidentally shifting to the left or right, does not negatively affect gameplay.
But the Leap Motion experience deteriorated when I switched to Touchless for Mac and replaced my mouse with my finger. Everything took five times longer and I began craving the touch of my Air's clickpad. Gestures worked great when the motion could be abstracted or visualized, but for fine motor skills I needed the friction and feedback of touch. A pen provided a greater amount of precision, but the lack of feedback still felt unnatural.
As I watched my coworkers waving their hands over the Leap Motion Controller, we discussed possible real-world applications for such a technology. A common suggestion was swiping through Netflix covers from your couch, but this technology has been available on the Kinect for a couple years without much pickup. Motion-controlled browsing hasn't caught on because it's not easier or more natural than using a remote. It's like trying eat a meal after your mouth has been shot full of novocaine.
A user interface that ignores human behavior does not improve how we interact
So where does that leave the Leap Motion Controller? It's in the middle of an evolutionary step in computer interaction, but doesn't quite provide the right solution. We're moving away from touch screens and buttons, but the replacement needs to be anchored in normal human behavior. Minority Report set certain expectations for a gesture-based interface, but now it's here, we see it's a technological step in the wrong direction.
It's important that we not lose track of the goal of technology: to make our lives easier. Human beings are meant to touch, manipulate, peel, push, click, tap, brush, rub, knock, squeeze, and feel. Creating a user interface that ignores human behavior does not improve how we interact.
The Leap Motion Controller, while cool, doesn't improve anything. And $79.99 is a lot to spend to impress your friends.