The game of Anti-Tetris is played by standing in front of a monitor and watch falling Tetris pieces overlaid on a video image of your body. Each hand is used to make pieces disappear so that they don’t stack up to the top of the screen. We don’t see this as the next big indie game. What we do see are some very interesting techniques for hand tracking.
An FPGA drives the game, using a camera as input. To track your hands the Cornell students figured out that YUV images show a specific range of skin tones which can be coded as a filter to direct cursor placement. But they needed a bit of a hack to get at those values. They patched into the camera circuit before the YUV is converted to RGB for the NTSC output.
Registering hand movement perpendicular to the screen is also a challenge that they faced. Because the hand location has already been established they were able to measure distance between the upper and lower boundaries. If that distance changes fast enough it is treated as an input, making the current block disappear.
Needs to work on that algorithm. Kinda crappy.
Hand tracking is more difficult than face tracking. Well done!
It’s not doing it geometrically, as it seems you might be thinking. Just locating somewhat appropriately sized blobs of fleshy colors. Probably wasn’t the best idea to wear a sweater with a near hand-sized and flesh-colored logo for the demo. ;)
Also, an error in Mike’s writeup. They didn’t patch into the YUV signal in the camera, but rather in the NTSC-to-VGA converter programmed in the FPGA.
Isn’t Anti-Tetris where you dissect a brick wall into various pieces in an anti-gravity environment?
To find his hands he can convert rbg color to hsv and find the zone where the color is similar to color of his skin using threshold and pixel checking…
He can tracks his hands using an a color tracking algorithm.
Convert RGB to HSV and threshold the image using HSV value and after find the white pixel on the binary image.