Finger Recognition On The Kinect

The Kinect is awesome, but if you want to do anything at a higher resolution detecting a person’s limbs, you’re out of luck. [Chris McCormick] over at CogniMem has a great solution to this problem: use a neural network on a chip to recognize fingers with hardware already connected to your XBox.

The build uses the very cool CogniMem CM1K neural network on a chip trained to tell the difference between counting from one to four on a single hand, as well as an ‘a-okay’ sign, Vulcan greeting (shown above), and rocking out at a [Dio] concert. As [Chris] shows us in the video, these finger gestures can be used to draw on a screen and move objects using only an open palm and closed fist; not too far off from the Minority Report and Iron Man UIs.

If you’d like to duplicate this build, we found the CM1K neural network chip available here for a bit more than we’d be willing to pay. A neural net on a chip is an exceedingly cool device, but it looks like this build will have to wait for the Kinect 2 to make it down to the consumer and hobbyist arena.

You can check out the videos of Kinect finger recognition in action after the break with World of Goo and Google Maps.

[youtube=http://www.youtube.com/watch?v=k0j1YsDmAXA&w=470]

[youtube=http://www.youtube.com/watch?v=ZOoxgtlqzhM&w=470]

9 thoughts on “Finger Recognition On The Kinect

  1. Hmm, apart from being an advert for the chip (which is pretty much expensive “unobtanium” anyway, “call for pricing” being a bad sign), the same application can be done in software. Neural net with 1024 neurons is not that computationally expensive, decent microcontroller can do it as well.

    The dedicated chip is perhaps a good solution for high speed processing when tied with something like a high-end FPGA (but then why not use the FPGA for it?), but it is an enormous overkill for something like 4 gestures with Kinect …

    1. >but it is an enormous overkill for something like 4 gestures with Kinect

      it barely does this with such a small number of neurons, limited to 16×16 picture and 3 layers.

    1. Dedicated hardware has utility, and this basically equates to speed. However, the key is the training and classification steps, with all the usual accuracy/specificity/flase-pos/neg, etc. issues.

      Calling these weighted gates neurons is OK, I guess.

      Old school cats can think of this as a monster comparator with up to N outputs matching one or more results. Thus “state” is replaced with “weight” and obviously you could tune the weights.

      I read the manual – it’s a nice bit of kit, especially since it does video decode on the chip. I am impressed!

      But again, the secret is in the training. If you don’t mind wasting the $$ and time to build a proper training program, you could basically churn out your own version of this using a video frame store and some dedicated memory – and now your production costs drop from $150 to $35-$50 per unit, at the expense of slightly larger size.

      After all, it’s basically a giant PROM. This topic is very interesting, and it would be interesting to play with an eval board. This is one of the most interesting chips ever featured on HAD.

  2. Well considering they could have had tech like the kinect back in the Late 90’s i guess the kinect 2 with these features will be out around 2030 or so.
    Hey, maybe we will all be too busy with the VR we were promised back in the 90’s to even worry about it.

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.