Kinect open source driver demo and hacking


The competition for the first Open Source driver for the Kinect is heating up.  [Marcan42] has released a driver that does video and depth.  He was able to do this without an Xbox and you can see it in action after the break. [LadyAda] has been hard at work as well, recording and dumping the data, and even writing a “hello world” that utilizes the motors in the Kinect.
We don’t know for sure how [Marcan42] recorded his data, but we can see [Ladyada] is using a high speed Beagle USB 480 to record the data going both ways between the Xbox360 and the Kinect.  That’s the kind of toy we would like to have sitting around. For those who don’t know what all the fuss is about, there’s a contest to see who can get an open source driver out there first. The prize has grown every time Microsoft says something bad about it.

[thanks Rapps]

Comments

  1. doragasu says:

    MARCAN GRANDE!

  2. Pix3l the B1t says:

    “…[Ladyada] has bee hard at work…”, you’re missing an ‘n’ :p
    This is awesome, I can’t wait until it’s fully interfaceable with computers :)

  3. Filespace says:

    Why does that look like a thermal imaging device…=)

  4. apophis says:

    wait, didn’t microsoft say they made it difficult to hack?

  5. EquinoXe says:

    @Filespace:
    “Why does that look like a thermal imaging device…=)”

    Because 3D data has to be translated to a 2D output.
    With thermal imaging the spectrum we can’t see must be translated to our visible spectrum.
    It is the translation to our visible spectrum that makes both look the same because it doesn’t correlate with the colors we normally associate to these images.

  6. Congratulations Marcan!

    Can’t wait to see the 3d scanners that are going to be made with it!

  7. chango says:

    Awesome work in just a few days!

    Next step is to turn that height field into a mesh and map the color camera onto it. Then it will be obvious what’s going on here.

  8. openkinect says:

    @marcan42 used the dumps from ladyada (adafruit). Adafruit got a beagle and started the process. I can only assume she was getting frustrated at how long this was taking.

  9. Squirrel says:

    lol first thing I thought when I saw that video was: Hey!! that’s my laptop

    on topic: Impressive. That was under a week since it was announced. Hope [Marcan42] puts the $$ to good use..

  10. NatureTM says:

    Wow, I thought it would take months. Nice work.
    Microsoft really could have made this more difficult. I wonder why they didn’t.

  11. starlino says:

    Stereo Vision can be implemented easily with 2 cheap usb camera and opencv (as explained for example here http://www.starlino.com/opencv_qt_stereovision.html), the fact that Kinect has the infrared and probably hardware processing is a great plus which is a bonus for many robotics applications.I will not be surprised if Microsoft releases a driver for their Robotics Studio soon…

  12. spyder_21 says:

    I think they made it easy due to the fact it would not hurt sales 1 bit, in fact increase them. Its not like coping games.

  13. alan says:

    You couldn’t get a camera stand?!

    good work, though.

  14. alan says:

    To me it seems like the reason that MS doesn’t want people to hack the hardware is that they want total control of their products. Why would the consumers buy a $60 game if they could download something as an opensource project. They have licensing obligations to provide to their game developers, insuring that people would actually buy their stuff. Most people that buy a kinect in regards to using it as a hacked device will most likely not buy the games. (not to say that some people still would though).

  15. DrAltaica says:

    >The competition for the first Open Source driver for the Kinect is heating up.

    >[Marcan42] has released a driver that does video and depth.

    >To demonstrate the driver you must also write an application with one “window” showing video (640 x 480) and one window showing depth.

    Ok so why isn’t the contest over?

  16. P says:

    @spyder_21
    the hardware is probably a loss leader, so increased sales of the kinect device without game sales will actually cost microsoft money

  17. rasz says:

    hah so it looks like cameras can be accessed with one of the V4L drivers, sweeeeet.

  18. Jack Sprat says:

    According to an article I read, Microsoft is more concerned with the software that runs the device and they say the most impressive part is the software they use to understand the 4 microphones. They are able to eliminate the echo of sound produced by the xbox. But the really cool part is that it can zero in on the voice of the player even when the room is full of noisy people and even follows the player as he/she moves. It’s always listening unlike other systems where you have to push a button to let it know it should be waiting for something important. I’m sure Microsoft put the tons of security in the software.

    The Kinect is so awesome. I see it as the beginning of a Star Trek like computer. “Computer, dim lights.”

  19. bitflusher says:

    as soon as the open source driver will be ported to the wii and fun homebrew things start to emerge i think microsoft will have a problem with this…

    will be fun though.

  20. Luiz Borges says:

    Haven’t seem the source code, but if it was easy to hack into it, it must be a very good design (from the interfacing stand point). That’s a plus, not a minus for the developers. If the result of your code is clear and understandable, you made a good work.

  21. Chris says:

    That was fast. Well done, Marcan42 and Ladyada!

    @Jack Sprat: I think the sound localization is easier than the skeletal tracking, and could be reproduced. I’ve seen some good papers on it. Basically you FFT the audio signal once to get the spectrum, which gives you a jumbled mess of frequencies and harmonics. Then run another FFT on the spectrum to get the cepstrum, which allows you to determine which harmonics are related, and therefore from the same sound source. Pick the loudest harmonic set that resembles the characteristics of a human voice. Then go back to the original FFT spectrum, and look at the phase difference for that harmonic set from different microphones; that gives you the speaker’s position. Once you have that, you have all the info you need to perform echo and noise reduction, basically all by removing frequencies that have phases which couldn’t have come from the speaker’s location. Finally, take the remaining frequency/phase info and do a reverse FFT to reconstruct the filtered audio. Any DSP gurus willing to take a shot?

  22. cgimark says:

    The SDK from the company that makes the hardware includes windows and linux drivers and from everything their sales department says its standard USB code for interfacing so nothing tricky involved. USB is a terrible thing to use if you want your hardware protocols kept quiet.

  23. Jonathan Wilson says:

    What we dont know yet about the audio is how much work is done by the audio hardware in the Kinect and how much is done on the 360 itself.

    Once someone figures out how to record the audio, we can see how much post processing needs to be applied to it.

  24. Whatnot says:

    On the subject of audio, you know; if you read the specs on your standard cheap realtek audio codec you’ll see they often have stereo mic inputs and clever algorithms to cancel echo and enhance sound directionally, it’s just that nobody uses it for some reason, apart from a rare notebook here and there maybe.

  25. error404 says:

    @spyder_21:

    The suggestion has been that making it difficult to hack was part of Microsoft’s agreement with PrimeSense. Those guys sell multi $k setups with similar capabilities to industry. An easy-to-interface baby version at an order of magnitude lower price might stand to cannibalize their market.

    How much truth there is to that we’ll likely never know. They may just try to use it to push copies of Win 8.

  26. strider_mt2k says:

    F MS for telling folks what to do with their toys after they’ve made their nut.

    Hack it hard.

  27. DrAltaica says:

    @Chris
    > I think the sound localization is easier than the skeletal tracking, and could be reproduced.

    I don’t know Chris, reproducing skeletal tracking is pretty easy…

    1. Go to http://i61www.ira.uka.de/users/knoop/VooDoo2/doc/html/index.html

    2. Download GPL source code

    3. enjoy!

  28. rasz says:

    I think Microsoft meant TORX screws when they said its tamper resistant :)

  29. Just take a look at the abundance of Amazon reviews for Kinect (mostly 5 stars). families are also uploading many Kinect videos on youtube. All smiles and laughs. You have to give MS their props. It’s a hit.

  30. xorpunk says:

    @starlino:also with variable geometry differentials like those used in modern digital camera algorithms, mostly by Sony. It’s actually not that hard, and can be done without IR or extra input channels like second camera.

    It’s what they use here except with IR like Sony cams do because it’s more reliable.

  31. bryce says:

    This will be sick if xbox takes it to the next level and makes a gun or some sort for call of duty and other games. So its all most like the movie gamer It would be history

  32. alan turing's dog says:

    Could we have a seperate category for Kinect hacks?

  33. Sunny Yang says:

    Hi, the video is wonderful.
    One newbie question:
    I’d like to save the distance information of dots scattered from the Kinect onto a moving object, flexible such as thin paper(size of Legal), in front of the Kinect sensor(IR sensor).

    In this case, do I need the Beagle USB480, suggested by ladyada, for data capturing?

    Thanks

    Sunny

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 92,449 other followers