The student projects that come out of [Bruce Land]’s microcontroller- and FPGA-programming classes feature here a lot, simple because some of them are amazing, but also because each project is a building-block for another. And we hope they will be for you.
This time around, [Junyin Chen] and [Ziqi Yang] created a five-in-a-row video game that is controlled by a pointing finger. A camera, pointed at the screen, films the player’s hand and passes the VGA data to an FPGA. And that’s where things get interesting.
An algorithm in the FPGA detects skin color and, after a few opening and closing operations, comes up with a pretty good outline of the hand. The fingertip localization is pretty clever. They sum up the number of detected pixels in the X- and Y-axis, and since a point finger is long and thin, locate the tip because it’s going to have a maximum value in one axis and a minimum along the other. Sweet (although the player has to wear long sleeves to make it work perfectly).
How does the camera not pick up the game going on in the background? They use a black-and-white game field that the skin-color detection simply ignores. And the game itself runs in a Nios embedded processor in the FPGA. There’s a lot more detail on the project page, and of course there’s a demo video below.
We love to follow along with Prof. Land‘s classes. His video series is invaluable, and the course projects have been an inspiration.
Neat idea, but it is a LOT easier to just add an IR frame around the display than it is to install a camera at a calibrated distance.
Now make this work with a cellphone tiny camera module at a very steep angle as in right on the frame of the monitor… Now you have something that has a real marketable value.
Seems like they could just subtract the background image in software rather than having to make it B&W.
This exactly how I would’ve tried to attack it, but the responsiveness or resolution might have to suffer for it.
Incredible. It takes a FPGA to know it’s getting the finger.
I wonder how it goes with different skin tones. As a species, we’re a varied lot with regard to colour, and early photographic film did a terrible job of photographing members of African tribes because the chemistry was tuned to European skin tones.
Good question, I did some research and it seems that we are all pink where it matters.
http://previews.123rf.com/images/slasny/slasny1305/slasny130500820/19767483-African-American-businessman-shaking-hands-with-caucasian-businessman-Stock-Photo.jpg
In the NIR, skin reflectivity is pretty uniform regardless of skin tone. Since these guys are doing edge detection anyway, it would be easy to just use a NIR source and ignore the color altogether.
Just peel the IR filter off their CCD camera?
That’s a good point James. Although it may be good to use visible and NIR alternately to help exclude non-human pointing devices.
Why? To do that would exclude the people with non human pointing devices.
Well if you insist on using a frozen stoat you can always paint it pink first.
It’s quite interesting.
What is that camera module used in this design?