Full of video and audio sensors, the newly released Kinect is Microsoft’s answer to Nintendo’s Wii MotionPlus and Sony’s PlayStation Move. Now there is money up for grabs to hack it. Adafruit is offering up a one thousand dollar prize to open source the driver for the Kinect. What do they want this driver to do? They want RGB and distance values. We’re excited to see the hacks that will come around because of this product, and now that prize money is involved, everything has been turned up a notch.
Update: The bounty has been raised to $2000 after a Microsoft response to CNET:
But Microsoft isn’t taking kindly to the bounty offer. “Microsoft does not condone the modification of its products,” a company spokesperson told CNET. “With Kinect, Microsoft built in numerous hardware and software safeguards designed to reduce the chances of product tampering. Microsoft will continue to make advances in these types of safeguards and work closely with law enforcement and product safety groups to keep Kinect tamper-resistant.”
Update: Progress toward a driver
Update: Winner of the Open Source Kinect contest
i got to play with one and its not all its hyped up to be
i prefer the playstation move XD
If I can’t play Halo or COD on it, whats the point? It’ll just be another wasted peripheral like the zapgun for NES.
It would be pretty amazing to have such a powerful and inexpensive device at our disposal. It’s just that $1000 may be nothing compared to the effort it could take. The Kinect uses about 10% of the XBox’s processing power. I’m guessing MS optimized the crap out of that 10%. If that’s all gesture recognition, then probably no big deal, but if the XBox handles the ranging, I don’t know…
I remember reading MS had all kinds of trouble developing this. Also, I’m sure they want to use the technology in other applications. I don’t know how hard it would be to recreate all that development, and I bet MS has taken steps to thwart our attempts.
Very nice of Adafruit though. I do think it will happen. It always does, but sometimes it takes a really long time.
Supposedly, in a few months MS should release interfaces to use Kinect with XNA Studio. Can you imagine how cheap would be a gesture/voice operated station? :)
I’d much rather prefer to have the person recognition systems in place too. Perhaps microsoft can offer a PC SDK with those amazing algorithms in place?
Shouldn’t there at least be a compatible open driver for the kinect? It’s based on a generic primasense device.
I’m sure MS stole it from an open source project, and will sue if you now imitate it.
I’m not even merely being snide.
Hey !
What about the Microphone Array driver too ? Kinect is also a great voice recognition device (like acoustic magic).
@NatureTM
If you read the description on the adafruit website, it says “the Kinect sensor outputs video at a frame rate of 30 Hz, with the RGB video stream at 32-bit color VGA resolution (640×480 pixels), and the monochrome video stream used for depth sensing at 16-bit QVGA resolution (320×240 pixels with 65,536 levels of sensitivity)”
So this information should be relatively easy to extract
But you never know…
in response to microsoft’s comment, the bounty has doubled to $2k
==========
But Microsoft isn’t taking kindly to the bounty offer. “Microsoft does not condone the modification of its products,” a company spokesperson told CNET. “With Kinect, Microsoft built in numerous hardware and software safeguards designed to reduce the chances of product tampering. Microsoft will continue to make advances in these types of safeguards and work closely with law enforcement and product safety groups to keep Kinect tamper-resistant.”
Read more: http://news.cnet.com/8301-13772_3-20021836-52.html#ixzz14MT9Zz00
I love this, they upped the bounty to $2k because M$ bragged abut the security in the device and their lawyers xD
Update: We’ve increased it to $2,000 – why? We just read this at CNET…
But Microsoft isn’t taking kindly to the bounty offer. Bounty offered for open-source Kinect driver – “Microsoft does not condone the modification of its products,” a company spokesperson told CNET. “With Kinect, Microsoft built in numerous hardware and software safeguards designed to reduce the chances of product tampering. Microsoft will continue to make advances in these types of safeguards and work closely with law enforcement and product safety groups to keep Kinect tamper-resistant.”
Don’t make us up’ it to $3k
Seems silly they would be worried about people using it on things other than the Xbox. I could see worrying about someone modifying the xbox/camera with a virus or something of that nature, so that its basically broadcasting a camera feed to the internet. But this is just giving people a reason to buy a microsoft product, who otherwise might not. I don’t own an xbox, but this gives me a reason to buy an xbox product. Sounds like a good deal for both sides.
Why would they want to stop people from hacking the Kinect? I wonder how many extra Wiimotes were sold because people were hacking them. Hell if someone hacks the Kinect and gets facial recognition and audio working, we’re talking HAL. Put a couple around the house and you’ve got a system that will do whatever you program it for.
When I heard this uses time of flight, my immediate reaction was that all of the underlying technology is heavily patented. That’s the reason why a SICK sensor still costs $5k. My theory is that primesense’s licensing requirements are very strict to keep this product out of the research/robotics domain. Thus, I wouldn’t be surprised to see a fully encrypted path between the kinect and the xbox. If true, this is very bad news for hacking. Due to the absence of any processor, the encryption would have to be built into the primesense IC which would make sniffing for keys really challenging.
The Kinect could be sold at a loss to drive up game sales, in which case buying it without the intention of buying XBox games would be bad for them.
And if you can learn how to extract data, you also learn how to insert data. Kinect cheat engine in the future?
But I think this is probably just the standard knee-jerk reaction large companies have when someone messes with their formats/protocols.
I can understand Microsoft wanting to protect its intellectual property as far as the software running on the Xbox that is able to do skeletal tracking from RGB and depth info, that’s fair in my book.
But even a dumb sensor that only returns a raw depth map is a huge advance to robotics.
Considering Microsoft bought out the only two companies that developed affordable gated time-of-flight depth sensing cameras, them taking measures to prevent people from purchasing and using that hardware for their own purposes is just bloody mean.
Kudos to Adafruit for offering the bounty. I hope this gets cracked wide open.
@Ice Brain
“The Kinect could be sold at a loss to drive up game sales, in which case buying it without the intention of buying XBox games would be bad for them.”
Ah yes, I totally forgot about this style of economics.
As far as their intellectual property, wouldn’t this be more of creating something new? Unless the people extract the software out of the xbox and copy paste it into their driver. It would seem to me all this is doing is creating a new interface to an existing product’s data stream. Like hooking up a vcr to your cable tv line instead of a cable box or something like that, not sure, am not a lawyer, just seems like this shouldn’t be that big of an issue, if nothing is extracted out of the xbox.
Does anyone know of a good site with driver programming tutorials?
I have no idea how they’re supposed to work or interface with their respective operating systems, but I’m a code wizard and I want to sniff me some USB.
http://oreilly.com/catalog/linuxdrive3/chapter/
I found this sample chapter fairly informative in regards to USB drivers. Along with libUSB taking care of all the down and dirty stuff.
Jan Axelson’s site http://www.lvr.com/ has a ton of good info too.
@M4CGYV3R
Try looking here: http://www.codeproject.com/KB/system/#Device+Drivers
They have some good tutorials and other related articles.
Instead, there should be a bounty to clone the Kinect with OpenCV and a pair of stereo cameras.
Probably be easier to build your own. I’ve never worked on an xbox before, no idea if you can run programs on it, otherwise wouldn’t this require a hardware USB sniffer? Those are pretty expensive from what I’ve seen. Not sure if you could build some pieces of equipment, a device and a host, to trick the kentic and xbox into giving up enough information to get the ball rolling on reversing the data.
Well, if $1000 wasn’t enough, apparantly it’s been upped to $2000. Now THAT should be enough for some real action to get started on this thing.
@whatnot You may remember the DARPA sponsored autonomous land navigation challenge a couplathree years ago. MS got the guys that placed 2nd and they developed the Natal/Kinect. Like Han says, “She’s got it where it counts kid.” It will be a nice piece of interface gear (minus the size I guess but ya mount it). Like others say, I’m excited but feel $1000 is too low a bounty unless they just want to get the first scoop on the hack which is what this reads more like. I shake my fists at the bloated God gizmodo and her phallic tentacles. User mileage may vary.
here is the teardown. http://www.ifixit.com/Teardown/Microsoft-Kinect-Teardown/4066/3
@blue carbuncle That might identify the people working on it but not necessarily where those got ‘inspiration’
“The Kinect could be sold at a loss to drive up game sales, in which case buying it without the intention of buying XBox games would be bad for them.”
This is exactly what MS did with the orignal Xbox. They sold the hardware at a loss, expected to make up for it in games and xbox live. So they went extreme lengths to secure it. They had a dummy bios on the mobo, and actually hid the bios inside the custom nvidia gpu. Since it was basically a legacy free PC, it meant people could buy it, hack it, install linux on it, and have a cheap ass pc.
Haha… I read “Hacking the Xbox” back in the day.
I’m all for hacking Kinect for the sake of hacking Kinect.
But I’ve seen the “sub-$100” RGB-D cameras. They work fine. There are a number of recent papers using them.
I don’t think you can buy them off the shelf just yet, so you might be able to save about a month if you hack Kinect really fast.
Why would they want to stop people from hacking the Kinect? I wonder how many extra Wiimotes were sold because people were hacking them. Hell if someone hacks the Kinect and gets facial recognition and audio working, we’re talking HAL. Put a couple around the house and you’ve got a system that will do whatever you program it for.
—————————-
1. Hacking works the other way too. Instead of increasing sales this reverse engineering could be used to make Kinect clones.
2. Perhaps the R&D and cost means they’re relying on game to break even, something PC users wouldn’t be buying.
Lately, MS has been saying that the kinect uses less than 10% of the xbox’s CPU power. If that’s true, do you really think it’s simply sending RGB distance values to the xbox? I bet it’s doing much more on-board processing, like real-time skeletal modeling to produce x, y, and z values (or difference vectors) for specific joints.
@NFN
I don’t think MS did the hardware, or at least I think they ported someone else’s reference design and used someone else’s RGB-D CCD. Going from there to a commercially viable product is a lot of work, especially since a lot of the magic is in the software that goes with it. But much of the “R” cost was elsewhere, while MS picked up the “D” portion related to what they wanted to use it for.
What does microsoft think Visual Studio does. it allows people to use their windows computers for something they were not designed to do. I guess if you are willing to pay $700-1000 for their software then its ok. When will these companies realize that if they make the code and hardware open then many more people will buy their products and probably buy multiples of them. I can’t even begin to guess how many PS3’s were bought just because they could run linux and how mad they were when they suddenly stopped working
From looking at the iFixit teardown its obvious that there is some kind of CPU or MCU in there
As for why MS doesnt want hacking, I am inclined to believe the “sell at a loss to make up money on games” and the “sell far below what similar systems would cost as long as its only used for games”
As for reverse engineering, as long as its not using encryption and as long as no (C) MS/(C) PrimeSense code is used, it should be legally clean.
Even if its using encryption, IIRC there are legal precedents (I cant find any right now though) where cracking encryption for interoperability like this is not a DMCA violation
I love how M$ talks about encryption and cant hack it, blah blah blah. Thats like the worst thing they could say.
Rule #1 Never tell a hacker they cant do something, unless you want them to do it.
If i had a XBOX i would start looking into it. Just for spite!
Isn’t the Kinect just a high-speed low-resolution camera? The console handles the processing.
Regarding any company trying to make juicy devices “tamper resistant” or “hack proof”:
Best of frakkin’ luck!
The original Xbox doesn’t even really wake up and rock until it’s properly hacked!
Duh!
The Kinect might not be using time of flight after all: http://www.pcmag.com/article2/0,2817,2365136,00.asp
“[…]Competing technologies use what’s known as “time of flight”, which projects out light and then measures the time it takes to get back to the sensor.[…]”
> Hacking works the other way too. Instead of increasing sales this reverse engineering could be used to make Kinect clones.
And I’m already wating for the moment to order a cheap kinect clone from dealextreme =)
A interesting look at the Kinect through a pair of IR goggles http://www.youtube.com/watch?v=r7nRKU0nFxA
Surely they were expecting it to be hacked, even if it uses encryption.
Although there’s no mention of I/O channel encryption circumvention exceptions in the Wikipedia DMCA article…
@adam: That’s exactly the point: until recently, every PS3 was a *loss* to Sony – they only made money through games. If you bought one to use as a Linux box, they’re were losing money.
And while using developing a Kinect driver for the PC is probably legal, a clone won’t be – they’ll infringe on a number of patents, either Microsoft’s or from the companies which produce it for them.
And no Chinese mock up company has the money to invest in the R&D required to clone each of those components. How many cheap Wii Remote clones that work with the Wii are there?
@adam
Or get one of the express editions of Visual Studio, for FREE.
Could it be the real reason why MS doesn’t want us hacking the device is that they’re selling them at a loss right now?
The Kinect tech has much more potential than games.
Facial recognition.
If they increase resolution this could be used for security.
These days it’s almost impossible to see what’s going on on every camera. Kinect could recognize violent behaviour and warn a security guard to take a look.
or maybe they don’t want it hacked because they’re planning on a PC based system that isn’t entirely based around gaming?
Whilst this looks good for gaming, it looks like it could be levaraged for some really nice minority report style interactive display technology.
I wonder if it would need anywhere near the pre-360 processing power if it were hooked up to a quad core cpu via usb3?
I am thinking M$ is wanting to use this device for passive monitoring of the fools that connect this thing up. Oh don’t hack it or you may find that code that sends your life to our Big Brother Network. Yeah I know conspiracy nut. BUT it does look like a fun project just to thumb a nose at Billy with.
I don’t think the $2000 is going to be taken anytime soon. I would be surprised if at least some of the Primesense IP was not running on the XBox’s processor, so I would expect the comms channel from the cameras to be well encrypted, and even then it might not contain readily intelligible data.
Perhaps the extra microphones are for doing beam forming for locating the voice in the room. I don’t recall the Primesense eval kit doing that and that IP certainly is in the public domain. Even so, this would make an expensive USB microphone.
“How many cheap Wii Remote clones that work with the Wii are there?”
Lots…
Pretty bad example :(
/me guesses that Marvell SoC has something to do with how it works… Kirkwood perhaps?
Microsoft bought 3DV Systems a while back. They were the ones who developed the camera technology. If anyone has drivers for the camera that they were going to release as a webcam, they are probably pretty close to what is used in the Kinect.
Microsft has said time and time again they bought 3dv just for the patents and that the tech they use is different.