While the venerable computer mouse dated before many of its users, the simplified two-dimensional pointing device still remains the de facto method of translating user input to computers. In the decades that followed, joysticks, foot pedals, trackpads, gyroscopic input devices, trackballs, pens and other unique peripherals have positioned themselves as possible alternatives.
Admittedly, many of these devices have been relegated to the area of special applications and extreme ergometry. In truth, only touchscreens and trackpads have proven themselves as alternatives with mass appeal, but even these popular devices are not suitable to replace the inexpensive mouse that is connected to desktops everywhere.
The Leap Motion Controller is another such attempt and an interesting one. The company is touting the device as a new paradigm for human input devices that allows users to wave, swipe, and browse through a digital world otherwise intended for keyboards, mice, and touchscreens.
What exactly is the Leap Motion Controller and how does it work?
The tiny device is hardly bigger than a USB stick. However, there are two cameras and three infrared LEDs in the Leap. With these components, the device can track hand and finger movements in all three spatial dimensions. We've seen similar technology applied elsewhere (e.g. Microsoft Kinect) so the jumping motion may seem a little less intriguing than usual. Still, using motion detection technology to interact with your computer is at least refreshing, if not refreshing magical.
It's hard not to draw parallels with existing technologies like Kinect, but it's worth noting that the Leap Motion works with remarkable precision. The control can detect the smallest movements (according to literature 1/100 millimeter) with little perceptible delay. Despite this impressive accuracy, the device is certainly not without its flaws. More on that in a moment …
First impressions and furnishings
The Leap Motion Controller comes in a small, sturdy box with a first-class feel. Inside you will find the controller base unit and two USB 3.0 cables of different lengths – one 6 feet long and the other 2 feet. Leap Motion did a good job of putting together something that feels high-end, despite its price tag below $ 80.
A paper manual was missing for the unit we received. However, owners who are just starting out should ensure their base is placed right in front of them with the shiny side up and the green LED facing them. Setup instructions are available to users visiting leapmotion.com/setup. This is essentially a link to the latest controller software.
After installing the software, users will be shown a demonstration to familiarize them with the device. It's not necessarily a tutorial. Rather, it is a tool designed to "inspire" you – spatial New Age music and a cool display of what the Leap Motion controller "sees" when you wave your hands. While it doesn't do much to show you how to use the device, this is where you start to see (or fantasize about) the potential of the jumping motion.
Ideally, the Leap Motion base is in front of your keyboard. Despite this suggestion, I found that placing it here resulted in unwanted input as I typed and fumbled at my desk. As a result, I placed the controller behind my keyboard to fix this problem. Of course, with longer periods of gesture it is a little more tiring to reach further outside, but the location was not entirely awkward or uncomfortable for occasional use.
I was asked to calibrate the device by pointing at a shiny surface and moving it around. If the device is moved and tilted, “paint the screen” until you get a good calibration rating. To be honest, I was a little confused. "Am I doing this right?" I wondered while moving the device in front of a mirror (Leap Motion says any reflective surface will do, but mirrors are ideal). Interestingly enough, I couldn't calibrate it with a mirror. After that failure, I was ready to give up my nerd badge. Instead, I pointed it at my 27-inch matte display and it worked like a charm. Imagine that. To be fair, the calibration seemed appropriate right away.
Another app-driven ecosystem
To use the device outside of the demo, users must visit Leap Motion's Airspace App Store. Since the gadget offers three-dimensional inputs, applications must be developed taking this function into account so that the jump movement can be used sensibly. It should be noted that the device can also be used as a direct mouse replacement. However, users should be warned: it is really frustrating.
Airspace is Leap Motion's attractive dashboard for managing apps and is available for both PC and Mac. It doesn't offer a lot of features or configuration, but it looks nice and does what it says it does. While installing apps was generally a one-click affair, I found that experiencing "dependencies" needs to be installed separately.
For example, when installing Cut the Rope, a warning was issued that Microsoft .NET 4.0 and XNA Framework 4.0 are required. Airspace will display a few links that you can click to download and install each package. What is particularly annoying, however, is that I have already installed these frameworks on my system. There may be technical or licensing reasons why XNA and .NET are not included, but I would like some kind of dependency detection and package bundling for a true one-click installation in the future.
The Leap Motion Control Panel offers a reasonable range of fine-tuning options, including adjusting the level of interaction and the accuracy / speed of tracking. I found "automatic" height (~ 22 cm) and "balanced" to be the best settings for me.
Right-clicking the system tray icon brings up the Leap Motion Visualizer so you can see some nifty wireframe hand action anytime. You can also pause and resume the chase.
Jump movement control
Using the controller seems pretty intuitive. The tiny Leap Motion base has a pretty wide field of view right above it. When you put your hand in pyramid view, a circle (or circles for multitouch) will appear on the screen. That way, you'll know where your hand and fingers are before interacting with anything on your screen. When you want to press or control something on your monitor, extend your hand and / or fingers forward until you reach an invisible barrier. This invisible “wall” is the threshold that you have to cross before the jumping movement registers your bumps and waves, clicks and swipes.
In theory, the jumping motion allows you to rotate, pinch, zoom, and move within three full dimensions. With that in mind, I thought Google Earth would be the ideal playground to test the device out, especially since Google added support for Leap Motion months ago.
After I started the app, I saw our beautiful planet earth – I just had to reach for it. So I did … and so I failed. The slightest twitch of my finger made the globe spin frantically, a movement that likely tossed all of its virtual residents into the dark darkness of space.
I stayed with it. In doing so, I discovered some helpful gestures – make a fist to stop spinning, flatten your hand to slow it down, and so on. I even started looking for gestures online. Ultimately, however, at no point did I feel as though I had mastered Google Earth with the Leap Motion. It's this experience that embodies my relationship with the device: it's incredibly neat, but flawed in that regard. Input / output isn't always what you intended, and this lack of absolute predictability occasionally creates frustration.
However, if we forget for a moment about the device's three-dimensional spatial awareness, Cut the Rope is actually a good example of what works on the device. The reason for this is that the controls are simple and there isn't much to interact with. All you have to do is swipe down some cartoon ropes very quickly. “Pressing” a virtual button in an app can be a little tricky as you need to drive a certain level of perfection so that you don't run the risk of not achieving your goal. While you're focused on the perfect push of a button like a laser beam, you also need to consider the invisible barrier – the wall that separates movement from action. Once you've crossed this imaginary barrier, your finger will begin to “click” and slide over things. However, without a really noticeable difference, it takes a little practice to determine where that barrier is.
Of course, while pressing buttons gets easier, there's no denying how difficult it is to get started. Some software titles (e.g. Cut The Rope, Frog Dissection, Corel Freestyle) recognize this problem and include an automatic selection feature that does the press for you when you hover over a button for a few seconds. I appreciate that and apps that include this definitely get a +1 on my book.
When I talked about Frog Dissection I was really disappointed with the lack of interactivity in this app. It may seem macabre (and naive), but I was really hoping to "peel back" the skin and dig into a frog squad with my bare hands. As you've since learned, it didn't work out exactly that way. The experience felt incongruous and deficient. While it's a good educational tool for learning about frog parts, it was a recommended choice and I don't think it really shows what's cool about the Leap Motion controller. It could have been made easier with a mouse or a touch screen and I don't feel like I have missed anything.
However, Boom Ball is a recommended choice that showed off Leap Motion's features very well. It's essentially a three-dimensional version of Breakout that adds (literally) depth to a classic game. It's not exactly Skyrim, but it plays almost like tennis – a player throws their ball forward, it breaks a few things, and it bounces back. If so, tilting your hand will also tilt the angle of the paddle, which will allow the ball to be lured to a desired location.
I was very interested in Corel Painter Freestyle, but found the user input way too sensitive and shaky. I'm not sure why, but I gave up because it was almost useless to me. Freestyle reportedly uses the depth of the jumping motion to create the illusion of pressure in addition to the usual X and Y values offered by computer mice and most capacitive touchscreens.
Use Leap Motion for Productivity … or not
As mentioned earlier, applications must be designed to take advantage of the controller's unique three-dimensional input. For software that isn't, there's always the "Touchless" app that runs in the background and allows you to swap your mouse for hand and finger gestures. The Leap Motion finally turns your PC into a touchscreen. So why not
At first I found Touchless to be an incredibly neat, but completely futile exercise. I struggled through the Windows 8 Metro interface in a somewhat frustrating experience. I accidentally managed to do all sorts of things that I didn't want to do. Although my first attempt at navigating Windows using Leap Motion was difficult, my apparent awkwardness managed to provoke some laughs from around me. Let's just be a long way from tampering with our computers a la Minority Report.
After trying old college a few times and revisiting Touchless, I've improved enough to be able to use a PC. The key is to be slow and very conscious. However, I admit it didn't take long for this to cause more trouble than it was worth. I will definitely stick with a mouse and keyboard for desktop and metro purposes.
Apparently, there are third-party utilities out there that can complement your Leap Motion experience. Although I haven't tried such programs on the PC, BetterTouchTool for Mac looks fascinating. I also found out that some people have also written their own programs to do some really cool things, like this person tearing it up in Borderlands 2, although admittedly after "many hours of practice".
I admit the Leap Motion Controller is something I want to fall in love with. It's just so cool. However, my experience with the device shows that for the time being at least, it is more of a neat conceptual “toy” than a practical device that you will get to every day. Your mileage can of course vary.
Take me with you, but one of the controller's biggest flaws seems to be its inability to both deliver 100 percent accuracy and read your mind at the same time. It is a very sensitive device and really picks up the smallest movements. I am really impressed with its precision. What it doesn't do, however, is correct my not-so-perfect 87-degree finger prick or smooth my heavily caffeinated, shaky hands. Actually yes – but not always. Sometimes it mistook one of my ankles for pointing; In other cases, it lost track of a finger because another of my digits overlapped vertically. It is this unpredictable but occasional input that frustrates the experience.
For the most part, the accuracy is there. To make the jumping movement really great it needs to have a better understanding of what you are trying to do. A good example is trying to press a software button in an app. I keep my hand very still and concentrate, I push myself forward as straight as possible and sometimes I can't help but miss the target. I am not a robot and I am not always ready to carefully and deliberately move like one. Unfortunately for the jumping movement, more times I remembered failing than I succeeded because it was easier to remember mistakes. It's impressive, but not perfect.
Then there is the whole thing: "What should I do with this thing ?!" Aspect of the leap controller. This mystery has also been posed by tablets, mice, and even the home PC itself (if you go back enough in time). I believe that more and more novel applications for the controller will emerge, especially given the infinite creativity of software and hardware manufacturers. However, at the moment there are a few apps that treat Leap Motion like a touchscreen. Unfortunately, in many cases, you wonder why you even use a Leap Motion controller.
It turns out that "thinking" in three dimensions is also a chore. As intuitive as the jumping movement may be and seems, there is a kind of hand-vision-mind learning curve that has to be overcome. However, you get used to a long enough period of time – I'm sure my experiences will only get better over time, although I don't suspect there is much room for much improvement.
After a few days, my experience with the Leap Motion Controller was peppered with side-by-side moments of joy, amazement and frustration. With the right app like Cut the Rope, Boom Ball, Fruit Ninja, or even Google Earth (if you can find out), the device is brilliantly fun. Sure, a mouse might be more accurate for some apps, but when it comes to casual gaming, the added challenge isn't really undesirable. With the wrong app, like trying to navigate your desktop with your fingers using Touchless or finger painting in Corel's hypersensitive Freestyle, the opposite has often happened.
Even so, the apps that get the most out of the Leap Motion controller seem to be games. The growing library of tablet-inspired games available through Airspace are generally forgiving enough and make use of the device's features to create a pleasantly unique experience. After all that has been said and done, the Leap Motion Controller is good for playing but bad for working.
Advantages: Impressive tracking capabilities for a first generation device. Small footprint. Unexpensive. Nice toy to play around with right now.
Disadvantage: Depending on the app, it can be frustrating to get things working. More suitable for simple gesture games. Poor performance in productivity scenarios.