Did Microsoft steal the Kinect?

In 2009, while Microsoft was busy designing and marketing what would become the Kinect, [Carlos Anzola], an inventor, tinkerer, and self-ascribed geek from Bogotá, Colombia, had been working for years on a nearly identical gesture interface for the PC. His creation, the Human interface Electronic Device, or HiE-D – pronounced ‘Heidi’ – was capable of gesture recognition years before Microsoft would release the Kinect.

After developing his gesture recognition device in 2007, Microsoft showed interest in [Carlos]‘ device – going so far as to request a prototype. Microsoft suggested that he should apply for a patent on his technology. [Carlos] did just that, sending in patent applications to both the United States Patent and Trademark Office and the World Intellectual Property Organization a mere two days before the announcement of Project Natal and a full seven months before Microsoft applied for their Kinect patent.

Since the release of the Kinect, [Carlos] has been showing the HiE-D around Bogotá and has put a few videos of his technology up on Youtube, one of which can be seen below. You can also check out his Youtube channel for some great demos.

With a reputation of experience in computer video, animation, and 3D processing, [Carlos] was approached by a technophobic American dentist who wanted a faster laser scanner for 3D modeling of his patient’s teeth. [Carlos] built an improved laser scanner and was featured on Caracol Noticias, a Columbian Newscast. [Carlos] had to deal with a bit of feature creep from the technophobe dentist, because he now wanted to manipulate the models of his patient’s mouths without using a mouse. [Carlos] was hugely influenced by the Minority Report computer interface, and decided the easiest way to interface with a computer would be a gesture interface.

[Carlos] set out to build a device that would allow a person to control a computer using only gestures. His HiE-D would be a surprisingly simple but devilishly clever device. The HiE-D projects a pattern of dots, or constellation, in infrared onto the user. Infrared is invisible to the human eye, but is easily picked up by a camera onboard the HiE-D. This is how the Microsoft Kinect works, and can be seen by a video camera in nightshot mode. When the camera detects a change in this constellation, the image is processed and can identify reference points in the user’s face, hands, or even their entire body.

After hearing of the HiE-D, Microsoft courted [Carlos] and requested a prototype. He gave Microsoft a prototype of the HiE-D, and according to [Carlos], it was taken to Redmond in February of 2007 – more than two years before the announcement of Project Natal. After meeting with Microsoft two more times that year, he was told by Microsoft that a patent on his invention wouldn’t be a bad idea.

While any action on Microsoft’s part would be speculation, we will say that the Kinect is remarkably similar to the HiE-D. Both use a ‘constellation’ of infrared dots projected on the user, and both can are able to detect the ‘skeleton’ of a user for motion control. The image below, from the HiE-D patent, shows how the movement of a face can be tracked.

Today, [Carlos] is in talks with a few interested companies that would like to produce the HiE-D. He says it would sell for only $50 USD, compared to $140 for a Microsoft Kinect. The HiE-D doesn’t have a camera to capture video of a user, so playing dress up with a HiE-D would be impossible. This wouldn’t be to much of a drawback, because some of the most impressive Kinect hacks we’ve seen wouldn’t change at all with the HiE-D.

[Carlos] says he’s been in contact with a few lawyers in Boston, who believes he has a case against Microsoft for patent infringement. He’s undecided about how to proceed at this point – Microsoft does have the war chest to go after Google for Android and defend itself over its use of 3D mapping. If we were [Carlos], we’d be pretty skeptical about our chances as well.

A flurry of interest from the hacker community surrounded the Kinect before its release date – there were bounties posted to develop an open-source driver so the Kinect would operate outside the closed Xbox ecosystem. The fact that a driver was released hours after the official launch of the Kinect is a testament to the interest in gesture recognition and the Minority Report interface. At Hack A Day, it’s not unusual to see tinkerers and geeks re-imagine existing products; there have been copies of the Microsoft Surface, and an attempt to reverse engineer the Playstation Move. Most of these are reimaginings of existing ideas or devices, which makes the uniqueness of [Carlos]‘s build all the more amazing.

We’re reminded of the abilities of the anonymous home tinkerer every day. To us, “hacker” is a label of creativity, investigation, and understanding. Like [Carlos], some of us eventually stumble upon a new idea that will change how humans interact with their environment. Although [Carlos] may not get the windfall he deserves, we’re still pretty jealous of his ability to build something, alone in a small workshop, that would change how people interact with computers.

Comments

  1. Aaron says:

    They bought it from a 3rd party company. /thread

  2. Sean says:

    Microsoft didn’t invent the Kinect sensor technology, PrimeSense did. http://www.primesense.com/

    Microsoft licenses the technology from PrimeSense. Although they did buy a competitor 3D camera company 3DV, which uses IR time of flight cameras – but mostly for patents.

    What Microsoft did do is create a very robust skeletal tracking system, a very robust directional microphone system, and applied these to their game console.

  3. earl says:

    if the story is true, sounds pretty cut and dry: Microsoft took it.

  4. J says:

    Well, that explains the Microsoft’s openness with the IP… it was not really theirs to begin with.

  5. Enzo says:

    I wonder why it does not surprise me that much?
    :-)

  6. RR says:

    Isn’t the Kinect’s technology patented by Primesense? I guess Microsoft looked at a bunch of different people’s stuff and integrated them together into Kinect.

  7. Max says:

    The only thing that surprises me here is that a techno*phobic* dentist (i.e. a dentist who is scared of technology, or wants to stay away from it) would contact a tinkerer and inventor.

  8. Leithoa says:

    R&D *does* stand for rip off and duplicated…

  9. BlueCoder says:

    The Kinect wasn’t original. The idea of gesture recognition has existed for a long time. No different than facial recognition or environment recognition. The only thing original was applying it to games and marketing it and Nintendo gets the credit for that.

  10. Kinkfisher says:

    It’s a bit more complicated than that. Microsoft’s contribution to the Kinect is in the software and algorithms that map the depth map from the IR camera to a human model, and translate the movement of the model into gestures to feed into apps. This is reflected in the claims in their patent application. The sensor itself is from PrimeSense.

    Carlos’s claims are more about the device as well as detecting gestures from IR depth map, but it does not mention a human model, i.e. His and Kinect’s are different approaches. As such his patent is more relevant to PrimeSense.

    Now the problem is, what Carlos’ patent describes had probably been done long before by PrimeSense and an erstwhile rival, Canesta, both of whom have a bunch of patents on that technology already. Canesta was also snapped up by Microsoft recently – most probably for its IP.

    As such, Carlos’s patent application is likely invalid because of the earlier patents from PrimeSense and Canesta. Hence, his chances seem slim to me.

  11. Jack Johnson says:

    or did Mircosoft Purchased the tech.

    http://www.metrolic.com/how-apple-missed-the-
    opportunity-to-buy-kinect-technology-144265/

  12. nrp says:

    If I’m not mistaken, most if not all of the patents related to the structured light 3D end of Kinect belong to Primesense, not Microsoft.

  13. Pete says:

    Microsoft didn’t develop the technology that drives the Kinect anyway. The IR dot pattern emitter and sensor were developed by an Israeli company called PrimeSense. Microsoft simply used their already-developed technology to make the Kinect.

    It seems a bit silly to claim that MS stole the idea from this guy when they openly admit to using tech developed by someone else to power their toy.

  14. rasz says:

    SOFTWARE patent = worthless

  15. t&p says:

    that music just does not match with that video

  16. CRJEEA says:

    His system seems to work a bit better than the Kinect. If he can produce them cheaper than Microsoft can produce the kinect maybe he could bring them out to allow gesture control of PCs to become a standard or as an open source device for the robotics community (:
    A few of these mounted on a wheelchair to achieve 3d mapping could be an interesting concept.
    Throughout history people have stolen ideas and bent patients if this is the case with the Kinect shame on you Microsoft.

  17. CRJEEA says:

    PS.
    This really does go to show that the hack of today truely does have the potential to be the world changing technology of tomorrow. (even if few people know it was you who sparked it all off first until years later)

  18. Steve says:

    Ersh that is painful to read.

    A structured light depth measurement system which looks slightly similar to the Kinect so therefore Microsoft must have stolen it. Seriously the article doesn’t even mention Primesense.

    So it tracks a skeleton…. thats something people have been trying to do for a long time. Just because they use a depth image means Microsoft stole.

    Article completely misses the point, maybe try googling primesense.

  19. Helb says:

    Look at Jeff Han’s work with multitouch. Months before MS Surface was announced.

  20. cpmike says:

    i think hes actually better off, from a business perspective. let microsoft get the masses/people familiar with the interface under the restriction of it being only for gaming. now all he needs to do is develop the appropriate API package, and he is free to sell his device to any of the number of companies dying to implement this interface commercially, but were restricted legally by MS. he’ll make a buttload…

  21. jos says:

    both ideas are similar, and maybe microsoft’s idea comes from this man’s design. But when we talk about patents, what is a copy, original idea, violation… it’s more much complicated. They are “only” ideas.

  22. The patent system has been coopted by the banks to create credit, so no matter who invents anything, eventually it will be patented in canada or the US and noone will hear about the original inventors.

  23. Old duffer says:

    For crying out loud. I saw an IR-based system similar to this demoed in 1995, running end-to-end on what would now be deemed a mid-range microcontroller.

    There’s nothing new under the sun, people.

  24. xorpunk says:
  25. xyros says:

    a salute to one of the few… who creates something out of nothing… :D

  26. torwag says:

    Well if he is lucky, Microsoft does what big companies do in such cases…

    they give him a few million dollar and he grants all rights to them. Whatever they offer it might be worse to consider to take it rather then feeding a bunch of lawyers over the next 5-8 years. If it goes to court, at the end everyone is rich and happy beside of Carlos for sure

  27. steve says:

    What a garbage. They didn’t steel it, because the idea is ubiquotus. Different people came to the same conclusion. If Carlos can’t market his invention but takes forever to develope: too bad for him. If Microsoft even tells him to apply for a patent! Or he shows it before patenting- i mean, come on! You get out what you put in.

  28. thekanester says:

    Seriously HAD, this article doesn’t demonstrate very good journalistic rigour, and is on the verge of being litigious. If I were you, I’d pull the title before MS get word of it.

  29. lexi says:

    I have worked with Panasonic’s d-Imager before (it uses similar tech). I know of at least 7 different companies that uses the same technology in their sensors. The gesture recognition software is arguably the biggest differentiator for Microsoft. They even said in an interview that protecting the ip contained in the software is more important than locking down the sensor. And I agree. A dept sensor alone is not worth much. Its Microsoft’s combination of innovations- the depth sensor, its pivoting base, the array microphone, the color camera, and especially the software that makes it better than any competing offering.

    We payed almost $3000 for the Panasonic d-imager at the start of the year. I think its simply amazing that you can get the Kinect for $150. And I’d rather have that, than some dude in colombia’s shitty prototype

  30. Bob says:

    If you actually read the patents they don’t even work the same way anyway! You can only patent a method because anyone can have an idea, it is making it work that is the hard part.

  31. I’m usually the last to defend Microsoft, but I thought they bought the connect technology from this company:

    http://news.cnet.com/New-game-controller-Your-hands/2100-1043_3-6222024.html

  32. KillerBug says:

    Actually, you can patent an idea, and people often do. An individual could never defend such a patent, but a large corporation of patent trolls can, even if the patent was issued years after that method was open source and public domain. Apple actually has a patent for using a touchscreen display as an input device (what else would you use it for?)…they got it a couple YEARS after HP released the iPaq with a touchscreen, and they are currently suing Samsung for “violating” this patent. HP never sued Apple because they didn’t own a patent either; they took the idea and methods from someone else who had taken them from someone else, and so on.

    Actually, an individual can’t defend any patent. It does not matter if they stole your source code letter for letter to the point that the “about” menu has your name in it or the PCBs have your copywright on them…huge corporations always win against the little guy with no capital to fight.

  33. @Climate Change Kills

    Where on earth did you get that idea? A hole in the ozone layer? The exact OPPOSITE happens, and not a thing to do with credit.

    Patent anything and the Chinese legally sell 99.999% of your design in western countries within a week. I know, I have three patents; one for an (OBD-1) EEC-IV handheld using 15 cents of electronics, the other two for companies I’ve worked for. They did this at least as far back as 1988, when I made the EEC-IV tester.

    Sears magically had one on their shelves that summer after I showed it to a CPA and the VP of Sears’ tool division/ex E.F. Hutton VP.

  34. uzerzero says:

    I like the fact that rather than making a big legal scandal out of this like the patent trolls that have been looking to make a quick buck, [Carlos] is offering up a comparison of his product to the Kinect and possibly producing it as a competitor. He says he’s undecided about bringing a case to Microsoft, to which I say he should be more worried about trying to protect HIE-D from Microsoft suing him in the future.

    It’s nice to have “homegrown” competitors to the big tech giants, if only they could stand their own.

  35. Alex says:

    Hey guys, I heard microsoft bought the kinect technology from primesense.

  36. blue carbuncle says:

    Um unless he invented a time machine where he won the DARPA land challenge then no, he didn’t invent the Kinect. What a retard for even trying to pull this. Everyday on HaD we see builds “similar” to other devices-that happens when you work with pre ordained components. This sad little person will swallow himself whole with his twisted view of reality. He should just chalk this up to life and try inventing a bulletproof meth dildo or a submarine made of cocaine, both of which would be highly prized in Bogota- if it hasn’t changed much in the 6 years since ive been there.

  37. Steve says:

    @ blue carbuncie Thats a bit harsh on poor Carlos Anzola, its not his fault that being able to read the wikipedia page on the kinect before writing an article on the internet is not mandatory. (see first line under technology heading)

  38. Thermos says:

    This is a stiff lesson to all hackers, engineers, inventors, and the like about keeping idea notebooks. If you have a notebook, esp. a hard bound one with acid free paper, with conceptual ideas dated before another group patented their idea you stand a good chance of invalidating the patent. Do not let the computer house all of your work. Keep a design journal and safe guard it with your life. Remember ink can be tested for age and is like its own timestamp; especially when there is a constant stream of entries into the journal with the same type of pen.

  39. steve says:

    poor judgment HAD, seriously. Why don’t you take this article down?

  40. Ru says:

    Microsoft looked at multiple cameras before settling on the primesense device which is what may have been infringing on this guy’s patents. I’m sure MS can argue that they licensed the hardware from primesense in good faith; for once, MS are not the bad guys here.

    Most of the clever work on the kinect is in the skeleton tracking software, anyway. The next gen kinect will use a different camera technology too. This guy isn’t really going to get anywhere with his complaints.

  41. dext3r says:

    @blue carbuncle: more info on said ‘bulletproof meth dildo’, please. thank you.

    lol

  42. Ru says:

    Whilst I’m grumbling, I’ll note the demo video shows neither face tracking nor gesture recognition. It uses some trivially simple position cues for the demo games of the sort that almost anyone could cobble together given a decent depth video source, of which there are many.

    Seems like his claims have little substance.

  43. xGROMx says:

    Not Microsoft, they would never use some one elses technology and call it their own…….

  44. Dave says:

    Once upon a time when DOS was king and Apple Computers were tinkering with a GUI interface, they opened their doors to Microsoft so they could gloat about the future of computer interfaces. Microsoft took a shit ton of notes and later developed the Mac OS into MS windows. The Mac OS was already stolen from Xerox by apple a short time earlier so it was just desserts really for Microsoft to return the favor.

    Either way, Gates has stolen in the past. He has stolen in the present. He will steal in the future.

  45. Yazoo says:

    Isn’t this the typical story though? Big money company comes out with a product that was either licensed, acquired, or developed in-house and then someone screams bloody murder about said big money company stealing their idea?

    Realistically speaking, multiple people can come up with the same idea sometimes and how they implement it depends on how successful the effort or end product is. Furthermore, as others have pointed out, this isn’t exactly a new and groundbreaking idea.

    So my advice to him is to have a little cheese with his whine :-)

  46. Kuhltwo says:

    When I worked for a Point of Sale software developer, we entered Micro-squirt’s software competition. One of their requirements to entering was the inclusion of your source code.

    I warned my boss about that, and he didn’t think they would “borrow” our code. Sure enough, 2 years later, they came out with their own POS software package. Even the user interface was the same.

  47. Cesar H. Osorio says:

    WELL I KNOW THAT IS TRUE BECOUSE I WAS WITH RAMIRO CARLO’S FRIEND, TALKING TO THE DENTIST, ACOUPLES YEARS AGO, IN MIAMI, SINCE THAT TIME I KNEW ABOUT CARLOS INVENTION, I’M LEAVING IN SOUTH FLORIDA FOR 11 YEARS.
    CESAR H. OSORIO

  48. Zeta says:

    It wouldn’t be a surprice if microsoft stole the idea and set up another company to save themselfs from the blame on pantent infrigement.

  49. Dan says:

    I agree with Yazoo. I mean Ive written plenty of scripts yet Ive seen plenty of them that are the exact same as what Ive created.
    Great minds do think alike.

  50. blackxino says:

    no surprise here as Microsoft love stealing things

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 92,324 other followers