Full of video and audio sensors, the newly released Kinect is Microsoft’s answer to Nintendo’s Wii MotionPlus and Sony’s PlayStation Move. Now there is money up for grabs to hack it. Adafruit is offering up a one thousand dollar prize to open source the driver for the Kinect. What do they want this driver to do? They want RGB and distance values. We’re excited to see the hacks that will come around because of this product, and now that prize money is involved, everything has been turned up a notch.
Update: The bounty has been raised to $2000 after a Microsoft response to CNET:
But Microsoft isn’t taking kindly to the bounty offer. “Microsoft does not condone the modification of its products,” a company spokesperson told CNET. “With Kinect, Microsoft built in numerous hardware and software safeguards designed to reduce the chances of product tampering. Microsoft will continue to make advances in these types of safeguards and work closely with law enforcement and product safety groups to keep Kinect tamper-resistant.”
Update: Progress toward a driver
Update: Winner of the Open Source Kinect contest
Haha the IR Projector spits out a bunch of IR beams. It’s literally a wii killer. Cant use a wiimote with the thing plugged in!
Interesting that IR projector sends dot pattern instead of lines/grid, also the dots didnt appear to be moving/changind/dynamic.
There is one hefty arm or the like doing a lot of work and a lot of image processing, the RAM clearly states that there is processing made to the image on site, so it might have nice data encryption algorithms inside, but there is also the advantage that if someone finds the JTAG pins then they can dump the flash.
“tamper-resistant”, yeah just like ‘they’ made anti-climb paint, that didn’t stop Banksy.
@Andrew
“When I heard this uses time of flight, my immediate reaction was that all of the underlying technology is heavily patented. That’s the reason why a SICK sensor still costs $5k. My theory is that primesense’s licensing requirements are very strict to keep this product out of the research/robotics domain.”
where did you get the idea that kinect uses Time Of Flight? kinect uses pattern projection, not TOF do determine distance.
And the patent isn’t the main reason laser rangefinders are expensive, it’s simply that you need equipment fast enough to detect the time it takes for light to travel a few feet.
@JimZ That was a fascinating youtube JimZ, very elucidating and interesting, thanks for the link.
addendum: I’d like to see a nightshot of that IR pattern from the point of the lens on the kinect, that might give even more info on what’s what.
iFixit did a teardown, but tomshardware had a nice summary.
http://www.tomsguide.com/us/Microsoft-Kinect-Teardown-iFixit-Project-Natal-xbox-360,news-8640.html
There’s 64MB DDR2 and a dedicated PrimeSense processor which “all the sensors are wired into [it] for processing before transmitting a refined depth map and color image to the Xbox.” Also, a fan and a 3 axis accelerometer, which seems surprising to me.
Just another example of how all Microsoft cares about is its self. Whats it matter to them if someone shells out the cash to buy one and then takes it apart so long as they don’t resell a re-purposed version thus infringing on their patent or copyright. Once it’s in the consumers hands it’s their property and they can do whatever they feel fit to with the device, so long as they don’t try making a warranty claim at some later point. Hell, Microsoft should be embracing this if you think about, once someone cracks it (and they will, it’s just a matter of time and sleepless nights) chances are more people, especially in the hacker community, will become interested in the Kinect. Instead of just 360 owners, Linux users for example, will end up buying the Kinect if they can put it to good use, thus creating an even larger profit for Microsoft. Hopefully Adafruits legal fees don’t end up driving the cost of their products up or putting them out of business.
Just a quick note to you guys –
I got a chance to talk to the guys who developed the kinect sensor at a hacker conference in israel (small startup named primesense) and they are releasing a PC version of the sensor. I assume it will be easier to hack than the xbox kinect, and they might be even willing to help out (they are pretty cool guys, and they attend hacker cons, so it’s worth a shot)
@Squirrel
your’e right the technology is not TOF, it’s called structured light. it’s pretty common actually, they just managed to get it to work well on a domestic setting. BTW, in 2009 microsoft spent 11 million $ buying an israeli company which maked TOF sensors for games and such, just to shut it down and bury the patents. so that’s thinking ahead for ya.
@M4C WHy not ask your professor that interned you to a university class when you were 8? SHouldn’t you know that already big guy? lma0
@tomas: The israeli company was bought for patents only. It has almost nothing to do with the current Kinect.
I was under the impression structured light needed few views at different angles to reconstruct static 3d object, but then again i found this in wiki
Spacecraft Hazard Avoidance Utilizing Structured Light
http://trs-new.jpl.nasa.gov/dspace/bitstream/2014/39375/1/05-3907.pdf
that is doing exactly what Kinect does, uses static laser projected dot pattern.
Fascinating stuff. That Primasense chip is doing a lot of work triangulating so many points. I wonder what is the actual delay, maybe that terrible 0.5 second lag we see in XBOX is caused by skeletal reconstruction.
how long will it take for someone to turn on one these cameras and capture viedo from you and post it?
microsoft has such a good track record,
blue screen of death, black screen of death, redring of death.
can’t wait to see what xxx screen od death happens with there phones.
PrimeSense is not tof. It projects a infrared speckle and measures with a conventional camera the size and shape of the individual dots to determine depth. No shutters or exotic physics here.
@rossum: Absolutely right. It seems the idea that it’s using time of flight came from the Wired article linked from the page on Adafruit. It’s totally wrong, yet is written as if it’s fact.
The reason this is cheap is because it’s using off-the-shelf sensors with narrow field of view. Then they motorized it to make it suitable for gaming by mechanically tracking the action. If you look at the video on the Primesense website, the users are stationary, sat on a couch directly in front of the TV.
Cheap is relative. There is a ridiculous amount of crap in this thing; a motor? a usb hub? a fan? If only it could be small and efficient …. like their software.
@rossum: lol! :-) I think MS must have got into the habit of putting extra fans in stuff now given their red ring of death experience.
By the way, when are you going to reveal the worlds cheapest TFT you promised at the end of your Nokia LCD post? I need small TFTs to make a pair of color shutter glasses.
this is good new!!
open it wide open!! it is pwning time!!
or we will ask PrimeSense for source code!!
lol
How it works
http://www.primesense.com/files/FMF_2.PDF
It really isn’t anything revolutionary. More getting it all to fit inside the package and at a decent price is what it unique. There is nothing special about the cameras used or any of the other hardware.
Since the device uses USB it will not take long to sniff all the traffic and get the protocols.
The xbox 360 sdk is also leaked so it will not be long.
Looking at the primesense web site.
The SDK for the chips shows:
* NITE 1.1 software package:
o Windows / Linux support
o Full documentation
o Comprehensive samples
o Demo package for reference
o Thin host-optimized algorithms
o Tested and approved gestures and UI controls
o Full-body games’ algorithms
* Reference Design 1.08 Development Unit:
o Fully functional, assembled reference design unit
o Depth image and audio outputs
o USB powered
o Windows / Linux drivers
so the drivers already exist you just need the sdk to get them :(
Someone got the motor and accelerometer going: http://nuigroup.com/forums/viewthread/11154/
For more detail on Primesense’s tech, don’t bother with the website. Look up their patents instead, far more informative. They list a few different methods of structured light depth sensing, some of which can be combined. It looks like someone might be able to DIY a depth sensor, although it would probably need a DSP or FPGA for decent speed.
Also, I found the following article:
http://www.winsupersite.com/win7/win8_leak.asp
Which suggests support for Kinect or something like it might be integrated into Windows 8. And that might explain why Microsoft doesn’t want anyone to figure out how to interface to the Kinect – they want us to buy a new OS version.
@Whatnot
>I’m sure MS stole it from an open source project,
I’m assuming you mean this open source project
http://wwwiaim.ira.uka.de/users/knoop/VooDoo/doc/html/
“The Kinect could be sold at a loss to drive up game sales”
Kinect is sold at a profit, straight from the horse’s mouth.
“I’m sure MS stole it from an open source project, ”
You’re also a retarded idiot. The technology in Kinect was developed with the help of two or three companies. Microsoft bought those companies. One is Canesta and the other is Primesense. This tech did not come from open source you stupid twat.
@joe – lolz
Well they just increased it to $3k….
Nothing illegal about using YOUR OWN HARDWARE for whatever you want. They wouldn’t complain if you were scooping dog poop with it.
There is nothing revolutionary here.
Has no one ever used OpenCV?
Instead of infrared, using two $5 webcams (in a stereo formation) from ebay and some openCV magic, i can get similar accurate results…why would you bother buying this product from the ‘world’s favorite company’…
This one is not as easy as reverse engineering the WiiMote, not even close – 2 cameras, 3 microphones, and motor control – all through USB. If you can not do the handshake, it will not send a thing back to you.
My prediction is, as long as nobody from inside leaks technical info, this thing will never be hacked. When the bounty goes to $20k in 6 months, somebody will say something.
Let’s see how it goes. It would just be a lot easier to buy your own inferred camera!!!
And Biscuit, that’s not true. They always have the right to sue even hacking your own hardware. Check your EULA somewhere in the manual, and you have already accepted it when you bought it in the first place.
i think one thing we’ve all missed here is that the kinect has facial recognition technology on the xbox, so MS are going to lose money by not releasing a universal driver for it, because i hate the concept of the kinect, but if i could use it for more than an xbox handsfree gaming experience, lets say, as a security camera for facial recognition for people entering the front door of my house then i’d buy one.
because let’s face it, it would make a good security system if done right:
1. Facial Recognition on entrance.
2. Voice Key
3. in public areas to recognize facial expressions for signs of agression between 2 people.
this thing has so much more potential than just a games controller, when will MS see this and start marketing it.