Kinect open source driver demo and hacking

The competition for the first Open Source driver for the Kinect is heating up.  [Marcan42] has released a driver that does video and depth.  He was able to do this without an Xbox and you can see it in action after the break. [LadyAda] has been hard at work as well, recording and dumping the data, and even writing a “hello world” that utilizes the motors in the Kinect.
We don’t know for sure how [Marcan42] recorded his data, but we can see [Ladyada] is using a high speed Beagle USB 480 to record the data going both ways between the Xbox360 and the Kinect.  That’s the kind of toy we would like to have sitting around. For those who don’t know what all the fuss is about, there’s a contest to see who can get an open source driver out there first. The prize has grown every time Microsoft says something bad about it.

[thanks Rapps]

33 thoughts on “Kinect open source driver demo and hacking

  1. @Filespace:
    “Why does that look like a thermal imaging device…=)”

    Because 3D data has to be translated to a 2D output.
    With thermal imaging the spectrum we can’t see must be translated to our visible spectrum.
    It is the translation to our visible spectrum that makes both look the same because it doesn’t correlate with the colors we normally associate to these images.

  2. lol first thing I thought when I saw that video was: Hey!! that’s my laptop

    on topic: Impressive. That was under a week since it was announced. Hope [Marcan42] puts the $$ to good use..

  3. To me it seems like the reason that MS doesn’t want people to hack the hardware is that they want total control of their products. Why would the consumers buy a $60 game if they could download something as an opensource project. They have licensing obligations to provide to their game developers, insuring that people would actually buy their stuff. Most people that buy a kinect in regards to using it as a hacked device will most likely not buy the games. (not to say that some people still would though).

  4. >The competition for the first Open Source driver for the Kinect is heating up.

    >[Marcan42] has released a driver that does video and depth.

    >To demonstrate the driver you must also write an application with one “window” showing video (640 x 480) and one window showing depth.

    Ok so why isn’t the contest over?

  5. According to an article I read, Microsoft is more concerned with the software that runs the device and they say the most impressive part is the software they use to understand the 4 microphones. They are able to eliminate the echo of sound produced by the xbox. But the really cool part is that it can zero in on the voice of the player even when the room is full of noisy people and even follows the player as he/she moves. It’s always listening unlike other systems where you have to push a button to let it know it should be waiting for something important. I’m sure Microsoft put the tons of security in the software.

    The Kinect is so awesome. I see it as the beginning of a Star Trek like computer. “Computer, dim lights.”

  6. Haven’t seem the source code, but if it was easy to hack into it, it must be a very good design (from the interfacing stand point). That’s a plus, not a minus for the developers. If the result of your code is clear and understandable, you made a good work.

  7. That was fast. Well done, Marcan42 and Ladyada!

    @Jack Sprat: I think the sound localization is easier than the skeletal tracking, and could be reproduced. I’ve seen some good papers on it. Basically you FFT the audio signal once to get the spectrum, which gives you a jumbled mess of frequencies and harmonics. Then run another FFT on the spectrum to get the cepstrum, which allows you to determine which harmonics are related, and therefore from the same sound source. Pick the loudest harmonic set that resembles the characteristics of a human voice. Then go back to the original FFT spectrum, and look at the phase difference for that harmonic set from different microphones; that gives you the speaker’s position. Once you have that, you have all the info you need to perform echo and noise reduction, basically all by removing frequencies that have phases which couldn’t have come from the speaker’s location. Finally, take the remaining frequency/phase info and do a reverse FFT to reconstruct the filtered audio. Any DSP gurus willing to take a shot?

  8. The SDK from the company that makes the hardware includes windows and linux drivers and from everything their sales department says its standard USB code for interfacing so nothing tricky involved. USB is a terrible thing to use if you want your hardware protocols kept quiet.

  9. What we dont know yet about the audio is how much work is done by the audio hardware in the Kinect and how much is done on the 360 itself.

    Once someone figures out how to record the audio, we can see how much post processing needs to be applied to it.

  10. On the subject of audio, you know; if you read the specs on your standard cheap realtek audio codec you’ll see they often have stereo mic inputs and clever algorithms to cancel echo and enhance sound directionally, it’s just that nobody uses it for some reason, apart from a rare notebook here and there maybe.

  11. @spyder_21:

    The suggestion has been that making it difficult to hack was part of Microsoft’s agreement with PrimeSense. Those guys sell multi $k setups with similar capabilities to industry. An easy-to-interface baby version at an order of magnitude lower price might stand to cannibalize their market.

    How much truth there is to that we’ll likely never know. They may just try to use it to push copies of Win 8.

  12. @starlino:also with variable geometry differentials like those used in modern digital camera algorithms, mostly by Sony. It’s actually not that hard, and can be done without IR or extra input channels like second camera.

    It’s what they use here except with IR like Sony cams do because it’s more reliable.

  13. Hi, the video is wonderful.
    One newbie question:
    I’d like to save the distance information of dots scattered from the Kinect onto a moving object, flexible such as thin paper(size of Legal), in front of the Kinect sensor(IR sensor).

    In this case, do I need the Beagle USB480, suggested by ladyada, for data capturing?



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s