Anti-Tetris Project Is A Study In Hand Tracking


The game of Anti-Tetris is played by standing in front of a monitor and watch falling Tetris pieces overlaid on a video image of your body. Each hand is used to make pieces disappear so that they don’t stack up to the top of the screen. We don’t see this as the next big indie game. What we do see are some very interesting techniques for hand tracking.

An FPGA drives the game, using a camera as input. To track your hands the Cornell students figured out that YUV images show a specific range of skin tones which can be coded as a filter to direct cursor placement. But they needed a bit of a hack to get at those values. They patched into the camera circuit before the YUV is converted to RGB for the NTSC output.

Registering hand movement perpendicular to the screen is also a challenge that they faced. Because the hand location has already been established they were able to measure distance between the upper and lower boundaries. If that distance changes fast enough it is treated as an input, making the current block disappear.

6 thoughts on “Anti-Tetris Project Is A Study In Hand Tracking

    1. It’s not doing it geometrically, as it seems you might be thinking. Just locating somewhat appropriately sized blobs of fleshy colors. Probably wasn’t the best idea to wear a sweater with a near hand-sized and flesh-colored logo for the demo. ;)

      Also, an error in Mike’s writeup. They didn’t patch into the YUV signal in the camera, but rather in the NTSC-to-VGA converter programmed in the FPGA.

  1. He can tracks his hands using an a color tracking algorithm.
    Convert RGB to HSV and threshold the image using HSV value and after find the white pixel on the binary image.

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.