Want Gesture-Tracking? All You Have To Do Is Lift Your Finger.

Watching Tony Stark wave his hands to manipulate projected constructs is an ever-approaching reality — at least in terms of gesture-tracking. Lift — a prototype built by a team from UC Irvine and FX Palo Alto Laboratory — is able to track up to ten fingers with 1.7 mm accuracy!

Lift’s gesture-tracking is achieved by using a DLP projector, two Arduino MKR1000s, and a light sensor for each digit. Lift’s design allows it to work on virtually any flat surface; the projected image acts as a grid and work area for the user. As their fingers move across the projected surface, the light sensors feed the information from the image to the Arduinos, which infers the location of each finger and translate it into a digital workspace. Sensors may also be mounted on other objects to add functionality.

So far, the team has used Lift as an input device for drawing, as well as using it to feign gesture controls on a standard laptop screen. The next step would be two or more projectors which would allow Lift to function fully and efficiently in three dimensions and directly interacting with projected media content. Can it also operate wirelessly? Yes. Yes, it can.

While we don’t have Tony Stark’s hologram workstation quite yet, we can still play Tetris, fly drones, and mess around with surgical robots.

12 thoughts on “Want Gesture-Tracking? All You Have To Do Is Lift Your Finger.

      1. As long as you remembered to lug along your DLP projector, have room to set it up, and have re calibrated the “gloves” VS. small device the plugs into USB and sits infront of or behind your keyboard

        I would have thought a 9dof IMU on each finger and one on the hand would have been easier. The IMU on the hand sets the base line values, and then the values from each finger are compared to the baseline and then the +/- would be used to determine finger position…

    1. The processing power required.

      This “Lift” thing’s Arduinos directly give the position data.

      Leap motion needs a powerful CPU (your computer) in addition to the sensor (stereoscopic camera) hardware.

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.