Real-Time FPGA Finger Detection

The student projects that come out of [Bruce Land]’s microcontroller- and FPGA-programming classes feature here a lot, simple because some of them are amazing, but also because each project is a building-block for another. And we hope they will be for you.

This time around, [Junyin Chen] and [Ziqi Yang] created a five-in-a-row video game that is controlled by a pointing finger. A camera, pointed at the screen, films the player’s hand and passes the VGA data to an FPGA. And that’s where things get interesting.

An algorithm in the FPGA detects skin color and, after a few opening and closing operations, comes up with a pretty good outline of the hand. The fingertip localization is pretty clever. They sum up the number of detected pixels in the X- and Y-axis, and since a point finger is long and thin, locate the tip because it’s going to have a maximum value in one axis and a minimum along the other. Sweet (although the player has to wear long sleeves to make it work perfectly).

How does the camera not pick up the game going on in the background? They use a black-and-white game field that the skin-color detection simply ignores. And the game itself runs in a Nios embedded processor in the FPGA. There’s a lot more detail on the project page, and of course there’s a demo video below.

We love to follow along with Prof. Land‘s classes. His video series is invaluable, and the course projects have been an inspiration.

12 thoughts on “Real-Time FPGA Finger Detection

  1. Neat idea, but it is a LOT easier to just add an IR frame around the display than it is to install a camera at a calibrated distance.

    Now make this work with a cellphone tiny camera module at a very steep angle as in right on the frame of the monitor… Now you have something that has a real marketable value.

  2. I wonder how it goes with different skin tones. As a species, we’re a varied lot with regard to colour, and early photographic film did a terrible job of photographing members of African tribes because the chemistry was tuned to European skin tones.

    1. In the NIR, skin reflectivity is pretty uniform regardless of skin tone. Since these guys are doing edge detection anyway, it would be easy to just use a NIR source and ignore the color altogether.

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.