Did Microsoft Steal The Kinect?

In 2009, while Microsoft was busy designing and marketing what would become the Kinect, [Carlos Anzola], an inventor, tinkerer, and self-ascribed geek from Bogotá, Colombia, had been working for years on a nearly identical gesture interface for the PC. His creation, the Human interface Electronic Device, or HiE-D – pronounced ‘Heidi’ – was capable of gesture recognition years before Microsoft would release the Kinect.

After developing his gesture recognition device in 2007, Microsoft showed interest in [Carlos]’ device – going so far as to request a prototype. Microsoft suggested that he should apply for a patent on his technology. [Carlos] did just that, sending in patent applications to both the United States Patent and Trademark Office and the World Intellectual Property Organization a mere two days before the announcement of Project Natal and a full seven months before Microsoft applied for their Kinect patent.

Since the release of the Kinect, [Carlos] has been showing the HiE-D around Bogotá and has put a few videos of his technology up on Youtube, one of which can be seen below. You can also check out his Youtube channel for some great demos.

With a reputation of experience in computer video, animation, and 3D processing, [Carlos] was approached by a technophobic American dentist who wanted a faster laser scanner for 3D modeling of his patient’s teeth. [Carlos] built an improved laser scanner and was featured on Caracol Noticias, a Columbian Newscast. [Carlos] had to deal with a bit of feature creep from the technophobe dentist, because he now wanted to manipulate the models of his patient’s mouths without using a mouse. [Carlos] was hugely influenced by the Minority Report computer interface, and decided the easiest way to interface with a computer would be a gesture interface.

[Carlos] set out to build a device that would allow a person to control a computer using only gestures. His HiE-D would be a surprisingly simple but devilishly clever device. The HiE-D projects a pattern of dots, or constellation, in infrared onto the user. Infrared is invisible to the human eye, but is easily picked up by a camera onboard the HiE-D. This is how the Microsoft Kinect works, and can be seen by a video camera in nightshot mode. When the camera detects a change in this constellation, the image is processed and can identify reference points in the user’s face, hands, or even their entire body.

After hearing of the HiE-D, Microsoft courted [Carlos] and requested a prototype. He gave Microsoft a prototype of the HiE-D, and according to [Carlos], it was taken to Redmond in February of 2007 – more than two years before the announcement of Project Natal. After meeting with Microsoft two more times that year, he was told by Microsoft that a patent on his invention wouldn’t be a bad idea.

While any action on Microsoft’s part would be speculation, we will say that the Kinect is remarkably similar to the HiE-D. Both use a ‘constellation’ of infrared dots projected on the user, and both can are able to detect the ‘skeleton’ of a user for motion control. The image below, from the HiE-D patent, shows how the movement of a face can be tracked.

Today, [Carlos] is in talks with a few interested companies that would like to produce the HiE-D. He says it would sell for only $50 USD, compared to $140 for a Microsoft Kinect. The HiE-D doesn’t have a camera to capture video of a user, so playing dress up with a HiE-D would be impossible. This wouldn’t be to much of a drawback, because some of the most impressive Kinect hacks we’ve seen wouldn’t change at all with the HiE-D.

[Carlos] says he’s been in contact with a few lawyers in Boston, who believes he has a case against Microsoft for patent infringement. He’s undecided about how to proceed at this point – Microsoft does have the war chest to go after Google for Android and defend itself over its use of 3D mapping. If we were [Carlos], we’d be pretty skeptical about our chances as well.

A flurry of interest from the hacker community surrounded the Kinect before its release date – there were bounties posted to develop an open-source driver so the Kinect would operate outside the closed Xbox ecosystem. The fact that a driver was released hours after the official launch of the Kinect is a testament to the interest in gesture recognition and the Minority Report interface. At Hack A Day, it’s not unusual to see tinkerers and geeks re-imagine existing products; there have been copies of the Microsoft Surface, and an attempt to reverse engineer the Playstation Move. Most of these are reimaginings of existing ideas or devices, which makes the uniqueness of [Carlos]’s build all the more amazing.

We’re reminded of the abilities of the anonymous home tinkerer every day. To us, “hacker” is a label of creativity, investigation, and understanding. Like [Carlos], some of us eventually stumble upon a new idea that will change how humans interact with their environment. Although [Carlos] may not get the windfall he deserves, we’re still pretty jealous of his ability to build something, alone in a small workshop, that would change how people interact with computers.

86 thoughts on “Did Microsoft Steal The Kinect?

  1. @dexter It seems perfect in that it can keep you awake for days, satisfied, and can even make for an uncomfortable seat in the winter time. I would like to clarify that it is more bullet “resistant” than proof, but every human comes with its own carrying case lol.

    @CESAR J CAPSLOCKIO Did you also hear that Ferris Bueller passed out in Baskin Robbins?
    You sound like a reasonable, if not credible source lol.

  2. Microsoft testing technology Hie-D for 2 weeks at its headquarters, One years before the Israeli company offers its products to them.
    I have some pictures.
    Thanks for your good comments!!!


  3. If you have seen the movie ”pirates of silicon valley” then it’s no wonder they steal from people (better steal a good idea than think of an bad one); ms-dos isn’t bill’s invention after all….and GUI was invented by Xerox and stolen by Apple……

  4. And now for a more serious comment.

    CARLOS ANZOLA wrote:
    “Microsoft testing technology Hie-D for 2 weeks at its headquarters, One years before the Israeli company offers its products to them.”

    If it was only around a year after you showed your hardware to Microsoft before Primesense showed theirs then how do you know that Primesense didn’t get their patents well before you did? It’s unlikely that an organized research and development company wouldn’t start the patent process for their technology BEFORE they approached a company like Microsoft. You’ve already stated that you didn’t try to get a patent until after Microsoft told you to.

    I am not a lawyer, but I don’t recommend that you comment on this yourself. Anything you say here could, possibly, effect any future court cases you find yourself in (including any possible cases where Primesense/Microsoft sues you if they think their patents came first and your invention steps on them.

  5. Like everyone else commenting here I can’t speak to the merits of the accusations, except if the on calling himself Carlos is Carlos. A call for HaD to pull this entry,seriously? About all we can expect from HaD are entries that they think may be of interest to a certain segment of the DIY community. We really expect them to have a staff of investigative reporters, or a lab to test every build they feature.

    @Thermos your suggestions are good ones for a variety of reasons. However a long ago article in a hobbyist electronics magazine details how they may be ineffective when there’s a gorilla in the battle, at that time one of the gorillas was Ma Bell. The gorillas will be granted access to all your notes, and work shop, you will not have access to none of the gorilla’s. While it may be cliche’s to say the little guy doesn’t stand a chance, it’s often true. Just try being the debtee in an estate settlement or in a bankruptcy settlement. Most likely copyrights, and patents are for those who have the financial means to pursue enforcing them in the civil courts. Not to mention gorillas have attorneys who report to work everyday to work for the gorilla.

    1. Re: pulling the article.

      why? I can’t vouch for Carlos’ story, but if any of you show up with a patent pre-dating someone elses and a good story to go along with it, we’ll probably publish that too. It is news, like it or not.

  6. My manager, in middle 1990’s, a college buddy of Bill Gates, would explain Gate’s philosophy as “Inventions do not exist.”

    I could tell you where the concept of drag and drop really came from and most importantly, why, ei. it’s purpose.

    My manager said to me one day, spying upon a competative technology from, I think we should call that the “thumbnail” view … once again lo and behold …

    “An invention by any other name is a Microsoft feature”


  7. It took many years in court to prove that Microsoft copied BASIC. By that time, one of the original authors was dead and the other had dropped out of the industry. Media won’t print the truth even when you prove it in court because Microsoft buys allot of advertising.

    It took many years in court to prove that Microsoft improperly bid on the DOS contract, bullied a competitor’s wife to sign a contract without a lawyer that allowed them to sell her husband’s product as their own and then improperly withheld royalty payments.

    Microsoft didn’t invent the web browser, spreadsheet, database, word processor or any of the products they made a fortune off of.

    Every one of them were invented by small developers and then copied by Microsoft.

  8. Skeleton tracking will be critically important for defensive purposes in the coming Zombie Apocalypse. Make sure you have a robust battery backup/solar charging system for your Kinect system, since regional power systems will undoubtedly fail.

  9. When Microsoft has made something by themselves??

    Starting with DOS, it is all bought!

    But crap! The difference between Win98 and Win7 (sucks) is that the new one has more colors! That is it!!!

    I hate’em!!!! I really really really hate’em!!

  10. @ Caleb See Patent US 7,433,024 B2, Oct. 7, 2008. I think thats another year before Carlos’s application, which describes the technology used and licensed from primesense by Microsoft.

  11. Steve.. Thanks for your comment.
    My first request of patent application was in May 29 2008, (date prior, in patent application)and the first show with the CEO of Microsoft (Bogota Colombia) was in July 2006. In this date they showed very much interest.
    I have some documents about our meeting, in this date.


  12. Microsoft has a long history of “laundering” intellectual property through 3rd parties. MSDOS, the software that Microsoft built its empire on, was purchased by MS as QDOS, from a company started by a disgruntled former Digital Research employee, and contained code developed by Digital Research as part of its CPM-86 OS.

    Internet Explorer was purchased from Spyglass and contained code from Mosaic. The list goes on and on.

    Don Lancaster, a long time inventor and technical writer often made the point that patents and copyrights only gvie the inventor the right to sue, but in the reality of a world dominated by multi billion dollar global corporations, whoever can pay for the most lawyers that wins. The little guy always loses.

    A better protection of an idea is to publish the idea into the public domain as widely an soon as possible, to minimize the damages from the mega-corporations when they abuse the legal system to steal intellectual property.

  13. It costs six or seven figure sums to bring ANY IP-based litigation against anybody. I think Carlos is in for years of frustration if he tries to go against the Borg Empire. He should have a good case, though, if he can come up with that >$1000/hour cost. Maybe he could solicit investors for THAT. Oh, guess not, that’s illegal.

    Welcome to America!

  14. Its surprising how many people think that Apple “stole the windowing concept from Micro$oft”

    Marketing is everything. If you have the spin figured out right on everything, you can even become President.

    Its the money!

  15. Countries like the Netherlands originally started because the people of the non-nether lands were so viciously oppressed that they could not make a living. So many of the thinking souls who were not involved in the corrupt power structure, all who were not the George Bush I or II, or Obama or Osama of their times.. who were not those or the scions of those in power, went west, to the edge of the land, and then further, to what was then the swamps, reclaimed unowned land and started their own country where heredity and the divine right of some was not the system of law and resourceful free men could prosper and thrive.

  16. Chris:
    You have a big Idea… we do it!!…

    The HIE-D technology have many opportunity in the World and in many camps.
    (Home, for example TV Control, interactive Toys, military, medical applications, safety, people with limited mobility,… and many more)

    I think the better way is follow in the work. however simultaneously I can claim my rights, with a legal good case..

    CARLOS ANZOLA (Inventor HiE-D Technlogy)

  17. I think some opinions here are a bit biased without real knowledge of the subject. I won’t pretend I know if Microsoft “stole” the Kinect or not, but as a researcher in machine learning, I do recall reading several articles that came out of Microsoft research from pre-2009(c.1999-2008) that had to do with lattice detection, contrast analysis, etc… so I wasn’t suprised to see them release the Kinect.

    I think it’s also important to note that their purchasing of the hardware from a third party doesn’t mean that they weren’t heavily involved in some if not all of the software and firmware development.

  18. I honestly don’t think this is an issue of IP theft, I think it’s an issue of convergent development. Carlos and PrimeSense happened to arrive at the same technological place around the same time. The key was visual processing capability.

    Even if it is, I doubt you would have much of a chance trying to sue Microsoft, and court and lawyer fees are going to cost you, and the payout won’t come for years. A lawsuit at this point is risking a lot with a very small chance of reasonable payout.

    A $50 Kinect minus the colored-light and audio? I know I’d be all over that like white on rice, as they say. I think the smart thing to do would be to market HiE-D as a competing product to the tinkerer and hacker demographic, and nobody has to get a bruised ego or sink his life savings into a futile court battle. Then, if Microsoft gets upset, they have to sue YOU, and this is the defender’s case.

  19. I also wanted to add: If Microsoft had not mass produced this technology, it would likely be far less accessible than it is today – before the Kinect, a similar peice of equipment would have set you back many thousands of dollars.

  20. This sounds like the plot to that horrible Ben Affleck movie, Paycheck.

    Here’s the problem…. you APPLIED for the patent before giving it to them….. did you GET the patent before giving it to them?


  21. Anyone that knows Microsoft’s history isn’t surprised by this. It’s their modus operandi. They steel a product or an idea, then run the inventor out of business, or force him to settle for a insultingly low amount.

    MS-BASIC (bastardized from mainframe original)
    MS-DOS (PCDOS anyone?)
    Spreadsheet software (EZCalc & Lotus 123)
    Windowing OS with icons (Mac OS which was legally based on Xerox technology)
    Disk compression (Ran Stacker out of business while fighting MS in court)
    MSWord (Word Perfect)
    Internet Explorer, which got them into trouble (Netscape, though granted, IE hand some Mosaic code licensed)
    etc. etc.

  22. In patent a word a little alter can make it unique, legally he may have nothing to fight.

    But I am sure, they took the idea from this and evolved it….

    Lesson to learn, so a next time. As to such algorithms, they have been on the study for more than 30 years…

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.