Did Microsoft steal the Kinect?

In 2009, while Microsoft was busy designing and marketing what would become the Kinect, [Carlos Anzola], an inventor, tinkerer, and self-ascribed geek from Bogotá, Colombia, had been working for years on a nearly identical gesture interface for the PC. His creation, the Human interface Electronic Device, or HiE-D – pronounced ‘Heidi’ – was capable of gesture recognition years before Microsoft would release the Kinect.

After developing his gesture recognition device in 2007, Microsoft showed interest in [Carlos]‘ device – going so far as to request a prototype. Microsoft suggested that he should apply for a patent on his technology. [Carlos] did just that, sending in patent applications to both the United States Patent and Trademark Office and the World Intellectual Property Organization a mere two days before the announcement of Project Natal and a full seven months before Microsoft applied for their Kinect patent.

Since the release of the Kinect, [Carlos] has been showing the HiE-D around Bogotá and has put a few videos of his technology up on Youtube, one of which can be seen below. You can also check out his Youtube channel for some great demos.

With a reputation of experience in computer video, animation, and 3D processing, [Carlos] was approached by a technophobic American dentist who wanted a faster laser scanner for 3D modeling of his patient’s teeth. [Carlos] built an improved laser scanner and was featured on Caracol Noticias, a Columbian Newscast. [Carlos] had to deal with a bit of feature creep from the technophobe dentist, because he now wanted to manipulate the models of his patient’s mouths without using a mouse. [Carlos] was hugely influenced by the Minority Report computer interface, and decided the easiest way to interface with a computer would be a gesture interface.

[Carlos] set out to build a device that would allow a person to control a computer using only gestures. His HiE-D would be a surprisingly simple but devilishly clever device. The HiE-D projects a pattern of dots, or constellation, in infrared onto the user. Infrared is invisible to the human eye, but is easily picked up by a camera onboard the HiE-D. This is how the Microsoft Kinect works, and can be seen by a video camera in nightshot mode. When the camera detects a change in this constellation, the image is processed and can identify reference points in the user’s face, hands, or even their entire body.

After hearing of the HiE-D, Microsoft courted [Carlos] and requested a prototype. He gave Microsoft a prototype of the HiE-D, and according to [Carlos], it was taken to Redmond in February of 2007 – more than two years before the announcement of Project Natal. After meeting with Microsoft two more times that year, he was told by Microsoft that a patent on his invention wouldn’t be a bad idea.

While any action on Microsoft’s part would be speculation, we will say that the Kinect is remarkably similar to the HiE-D. Both use a ‘constellation’ of infrared dots projected on the user, and both can are able to detect the ‘skeleton’ of a user for motion control. The image below, from the HiE-D patent, shows how the movement of a face can be tracked.

Today, [Carlos] is in talks with a few interested companies that would like to produce the HiE-D. He says it would sell for only $50 USD, compared to $140 for a Microsoft Kinect. The HiE-D doesn’t have a camera to capture video of a user, so playing dress up with a HiE-D would be impossible. This wouldn’t be to much of a drawback, because some of the most impressive Kinect hacks we’ve seen wouldn’t change at all with the HiE-D.

[Carlos] says he’s been in contact with a few lawyers in Boston, who believes he has a case against Microsoft for patent infringement. He’s undecided about how to proceed at this point – Microsoft does have the war chest to go after Google for Android and defend itself over its use of 3D mapping. If we were [Carlos], we’d be pretty skeptical about our chances as well.

A flurry of interest from the hacker community surrounded the Kinect before its release date – there were bounties posted to develop an open-source driver so the Kinect would operate outside the closed Xbox ecosystem. The fact that a driver was released hours after the official launch of the Kinect is a testament to the interest in gesture recognition and the Minority Report interface. At Hack A Day, it’s not unusual to see tinkerers and geeks re-imagine existing products; there have been copies of the Microsoft Surface, and an attempt to reverse engineer the Playstation Move. Most of these are reimaginings of existing ideas or devices, which makes the uniqueness of [Carlos]‘s build all the more amazing.

We’re reminded of the abilities of the anonymous home tinkerer every day. To us, “hacker” is a label of creativity, investigation, and understanding. Like [Carlos], some of us eventually stumble upon a new idea that will change how humans interact with their environment. Although [Carlos] may not get the windfall he deserves, we’re still pretty jealous of his ability to build something, alone in a small workshop, that would change how people interact with computers.

Comments

  1. blue carbuncle says:

    @dexter It seems perfect in that it can keep you awake for days, satisfied, and can even make for an uncomfortable seat in the winter time. I would like to clarify that it is more bullet “resistant” than proof, but every human comes with its own carrying case lol.

    @CESAR J CAPSLOCKIO Did you also hear that Ferris Bueller passed out in Baskin Robbins?
    You sound like a reasonable, if not credible source lol.

  2. macona says:

    @Dave

    Apple did not steal the GUI from parc, Xerox gave access to it for 100,000 shares apple stock.

  3. Colecoman1982 says:

    Hey guys, I’m surprised nobody’s brought up the fact that Microsoft purchased the technology behind the Kinect from an Israeli company called Primesense.

  4. asheets says:
  5. Carlos A says:

    Microsoft testing technology Hie-D for 2 weeks at its headquarters, One years before the Israeli company offers its products to them.
    I have some pictures.
    Thanks for your good comments!!!

    CARLOS ANZOLA (Inventor the HiE-D TECHNOLOGY )

  6. Wilfred1970 says:

    If you have seen the movie ”pirates of silicon valley” then it’s no wonder they steal from people (better steal a good idea than think of an bad one); ms-dos isn’t bill’s invention after all….and GUI was invented by Xerox and stolen by Apple……

  7. Colecoman1982 says:

    And now for a more serious comment.

    CARLOS ANZOLA wrote:
    “Microsoft testing technology Hie-D for 2 weeks at its headquarters, One years before the Israeli company offers its products to them.”

    If it was only around a year after you showed your hardware to Microsoft before Primesense showed theirs then how do you know that Primesense didn’t get their patents well before you did? It’s unlikely that an organized research and development company wouldn’t start the patent process for their technology BEFORE they approached a company like Microsoft. You’ve already stated that you didn’t try to get a patent until after Microsoft told you to.

    I am not a lawyer, but I don’t recommend that you comment on this yourself. Anything you say here could, possibly, effect any future court cases you find yourself in (including any possible cases where Primesense/Microsoft sues you if they think their patents came first and your invention steps on them.

  8. D_ says:

    Like everyone else commenting here I can’t speak to the merits of the accusations, except if the on calling himself Carlos is Carlos. A call for HaD to pull this entry,seriously? About all we can expect from HaD are entries that they think may be of interest to a certain segment of the DIY community. We really expect them to have a staff of investigative reporters, or a lab to test every build they feature.

    @Thermos your suggestions are good ones for a variety of reasons. However a long ago article in a hobbyist electronics magazine details how they may be ineffective when there’s a gorilla in the battle, at that time one of the gorillas was Ma Bell. The gorillas will be granted access to all your notes, and work shop, you will not have access to none of the gorilla’s. While it may be cliche’s to say the little guy doesn’t stand a chance, it’s often true. Just try being the debtee in an estate settlement or in a bankruptcy settlement. Most likely copyrights, and patents are for those who have the financial means to pursue enforcing them in the civil courts. Not to mention gorillas have attorneys who report to work everyday to work for the gorilla.

    • Caleb Kraft says:

      Re: pulling the article.

      why? I can’t vouch for Carlos’ story, but if any of you show up with a patent pre-dating someone elses and a good story to go along with it, we’ll probably publish that too. It is news, like it or not.

  9. anymouse says:

    My manager, in middle 1990’s, a college buddy of Bill Gates, would explain Gate’s philosophy as “Inventions do not exist.”

    I could tell you where the concept of drag and drop really came from and most importantly, why, ei. it’s purpose.

    My manager said to me one day, spying upon a competative technology from, I think we should call that the “thumbnail” view … once again lo and behold …

    “An invention by any other name is a Microsoft feature”

    anymouse

  10. Harvey Birdman says:

    It took many years in court to prove that Microsoft copied BASIC. By that time, one of the original authors was dead and the other had dropped out of the industry. Media won’t print the truth even when you prove it in court because Microsoft buys allot of advertising.

    It took many years in court to prove that Microsoft improperly bid on the DOS contract, bullied a competitor’s wife to sign a contract without a lawyer that allowed them to sell her husband’s product as their own and then improperly withheld royalty payments.

    Microsoft didn’t invent the web browser, spreadsheet, database, word processor or any of the products they made a fortune off of.

    Every one of them were invented by small developers and then copied by Microsoft.

  11. James Rott says:

    60+ comments in…

    Sometimes I wonder how such a technical community can be so obtuse to reading or comprehending, editors included.

  12. blodgar says:

    Skeleton tracking will be critically important for defensive purposes in the coming Zombie Apocalypse. Make sure you have a robust battery backup/solar charging system for your Kinect system, since regional power systems will undoubtedly fail.

  13. Daniel says:

    When Microsoft has made something by themselves??

    Starting with DOS, it is all bought!

    But crap! The difference between Win98 and Win7 (sucks) is that the new one has more colors! That is it!!!

    I hate’em!!!! I really really really hate’em!!

  14. Steve says:

    @ Caleb See Patent US 7,433,024 B2, Oct. 7, 2008. I think thats another year before Carlos’s application, which describes the technology used and licensed from primesense by Microsoft.

  15. Carlos A says:

    Steve.. Thanks for your comment.
    My first request of patent application was in May 29 2008, (date prior, in patent application)and the first show with the CEO of Microsoft (Bogota Colombia) was in July 2006. In this date they showed very much interest.
    I have some documents about our meeting, in this date.

    CARLOS ANZOLA (Inventor HiE-D TECHNOLOGY )

  16. Nik P says:

    Microsoft has a long history of “laundering” intellectual property through 3rd parties. MSDOS, the software that Microsoft built its empire on, was purchased by MS as QDOS, from a company started by a disgruntled former Digital Research employee, and contained code developed by Digital Research as part of its CPM-86 OS.

    Internet Explorer was purchased from Spyglass and contained code from Mosaic. The list goes on and on.

    Don Lancaster, a long time inventor and technical writer often made the point that patents and copyrights only gvie the inventor the right to sue, but in the reality of a world dominated by multi billion dollar global corporations, whoever can pay for the most lawyers that wins. The little guy always loses.

    A better protection of an idea is to publish the idea into the public domain as widely an soon as possible, to minimize the damages from the mega-corporations when they abuse the legal system to steal intellectual property.

  17. Chris says:

    It costs six or seven figure sums to bring ANY IP-based litigation against anybody. I think Carlos is in for years of frustration if he tries to go against the Borg Empire. He should have a good case, though, if he can come up with that >$1000/hour cost. Maybe he could solicit investors for THAT. Oh, guess not, that’s illegal.

    Welcome to America!

  18. Chris says:

    Its surprising how many people think that Apple “stole the windowing concept from Micro$oft”

    Marketing is everything. If you have the spin figured out right on everything, you can even become President.

    Its the money!

  19. Chris says:

    Thats a good idea.. Carlos, why don’t you partner with Apple, and then let Micro$oft try (and fail) to sue them!

    Better to be defending than (trying to) attack.

  20. Chris says:

    Countries like the Netherlands originally started because the people of the non-nether lands were so viciously oppressed that they could not make a living. So many of the thinking souls who were not involved in the corrupt power structure, all who were not the George Bush I or II, or Obama or Osama of their times.. who were not those or the scions of those in power, went west, to the edge of the land, and then further, to what was then the swamps, reclaimed unowned land and started their own country where heredity and the divine right of some was not the system of law and resourceful free men could prosper and thrive.

  21. Carlos A says:

    Chris:
    You have a big Idea… we do it!!…

    The HIE-D technology have many opportunity in the World and in many camps.
    (Home, for example TV Control, interactive Toys, military, medical applications, safety, people with limited mobility,… and many more)

    I think the better way is follow in the work. however simultaneously I can claim my rights, with a legal good case..

    CARLOS ANZOLA (Inventor HiE-D Technlogy)

  22. Chris Marshall says:

    Get real! Does anyone know of anything worthwhile that Microsoft did not steal?

  23. Jordan says:

    I think some opinions here are a bit biased without real knowledge of the subject. I won’t pretend I know if Microsoft “stole” the Kinect or not, but as a researcher in machine learning, I do recall reading several articles that came out of Microsoft research from pre-2009(c.1999-2008) that had to do with lattice detection, contrast analysis, etc… so I wasn’t suprised to see them release the Kinect.

    I think it’s also important to note that their purchasing of the hardware from a third party doesn’t mean that they weren’t heavily involved in some if not all of the software and firmware development.

  24. Volfram says:

    I honestly don’t think this is an issue of IP theft, I think it’s an issue of convergent development. Carlos and PrimeSense happened to arrive at the same technological place around the same time. The key was visual processing capability.

    Even if it is, I doubt you would have much of a chance trying to sue Microsoft, and court and lawyer fees are going to cost you, and the payout won’t come for years. A lawsuit at this point is risking a lot with a very small chance of reasonable payout.

    A $50 Kinect minus the colored-light and audio? I know I’d be all over that like white on rice, as they say. I think the smart thing to do would be to market HiE-D as a competing product to the tinkerer and hacker demographic, and nobody has to get a bruised ego or sink his life savings into a futile court battle. Then, if Microsoft gets upset, they have to sue YOU, and this is the defender’s case.

  25. Jordan says:

    I also wanted to add: If Microsoft had not mass produced this technology, it would likely be far less accessible than it is today – before the Kinect, a similar peice of equipment would have set you back many thousands of dollars.

  26. Lauren Glenn says:

    This sounds like the plot to that horrible Ben Affleck movie, Paycheck.

    Here’s the problem…. you APPLIED for the patent before giving it to them….. did you GET the patent before giving it to them?

    Lauren

  27. Steven Jeter says:

    Holy shit. I should have seen this coming. A large enterprice stealing from the smaller man. How did I miss this.

  28. SuperNuRd says:

    This just in did Asus steal the Xtion Pro.
    Answer no don’t be stupid just because it has a patent doesn’t mean you can”t make something similar!

  29. Loke says:

    That’s how Steve Ballmer remains in position!

  30. Steven Crowley says:

    Really “Did someone steal this phrase from me”

  31. Carlos A says:

    Hi, everybody.

    If anybody want write me , I can answer your questions in this email :
    carlitos1904@gmail.com

    Thanks…

    CARLOS ANZOLA (Inventor technology HiE-D)

  32. SuperSparky says:

    Anyone that knows Microsoft’s history isn’t surprised by this. It’s their modus operandi. They steel a product or an idea, then run the inventor out of business, or force him to settle for a insultingly low amount.

    MS-BASIC (bastardized from mainframe original)
    MS-DOS (PCDOS anyone?)
    Spreadsheet software (EZCalc & Lotus 123)
    Windowing OS with icons (Mac OS which was legally based on Xerox technology)
    Disk compression (Ran Stacker out of business while fighting MS in court)
    MSWord (Word Perfect)
    Internet Explorer, which got them into trouble (Netscape, though granted, IE hand some Mosaic code licensed)
    etc. etc.

  33. X says:

    In patent a word a little alter can make it unique, legally he may have nothing to fight.

    But I am sure, they took the idea from this and evolved it….

    Lesson to learn, so a next time. As to such algorithms, they have been on the study for more than 30 years…

  34. Ur mama says:

    Playstaion eye does that too but i dont think they ever got any information about things like this but microsoft did.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 92,374 other followers