Anti-Tetris project is a study in hand tracking

anti-tetris

The game of Anti-Tetris is played by standing in front of a monitor and watch falling Tetris pieces overlaid on a video image of your body. Each hand is used to make pieces disappear so that they don’t stack up to the top of the screen. We don’t see this as the next big indie game. What we do see are some very interesting techniques for hand tracking.

An FPGA drives the game, using a camera as input. To track your hands the Cornell students figured out that YUV images show a specific range of skin tones which can be coded as a filter to direct cursor placement. But they needed a bit of a hack to get at those values. They patched into the camera circuit before the YUV is converted to RGB for the NTSC output.

Registering hand movement perpendicular to the screen is also a challenge that they faced. Because the hand location has already been established they were able to measure distance between the upper and lower boundaries. If that distance changes fast enough it is treated as an input, making the current block disappear.

Comments

  1. Needs to work on that algorithm. Kinda crappy.

  2. toodlestech says:

    Hand tracking is more difficult than face tracking. Well done!

    • Chris C. says:

      It’s not doing it geometrically, as it seems you might be thinking. Just locating somewhat appropriately sized blobs of fleshy colors. Probably wasn’t the best idea to wear a sweater with a near hand-sized and flesh-colored logo for the demo. ;)

      Also, an error in Mike’s writeup. They didn’t patch into the YUV signal in the camera, but rather in the NTSC-to-VGA converter programmed in the FPGA.

  3. Ren says:

    Isn’t Anti-Tetris where you dissect a brick wall into various pieces in an anti-gravity environment?

  4. DLion says:

    To find his hands he can convert rbg color to hsv and find the zone where the color is similar to color of his skin using threshold and pixel checking…

  5. DLion says:

    He can tracks his hands using an a color tracking algorithm.
    Convert RGB to HSV and threshold the image using HSV value and after find the white pixel on the binary image.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 93,941 other followers