Get digital plastic surgery thanks to openFrameworks and some addons

[Kyle McDonald] is trying out a new look, at least in the digital world, with the help of some openFrameworks video plugins. He’s working with [Arturo Castro] to make real-time facial substitution as realistic as possible. You can see that [Arturo's] own video has a different take on shading and color of the facial alterations that makes them a bit less realistic than what [Kyle] was able to accomplish (see that clip after the break).

The setup depends on some facial tracking software developed by [Jason Saragih]. That package is wrapped in ofxFaceTracker (already linked at the top of this article) which makes it play nicely with openFrameworks. From there, it’s just a matter of image processing. If you think you’re up to the challenge, grab your own copies of the source code and get to work. We’re shocked by how real this looks, even when [Kyle] grabs his cheeks and stretches them out. If someone can fix some of the artifacts around the edges of the sampled faces this would be ready to use when video-conferencing.

It kind of makes us think of technology seen in The Running Man.

[Thanks Luke]


  1. Max says:

    This is freaking awesome!

  2. Chad says:

    this is just strange and weird !!! gives me the heebee geebees!!

    interesting but strange!!

  3. MS3FGX says:

    Well, this is officially the most disturbing thing I’ve seen today.

  4. Mike says:

    MAN that picture gave me the creeps

  5. zing says:

    My brain is screaming now.

  6. biozz says:

    WOAH thats creepy

  7. Greg says:

    I could have gone my whole life without seeing that picture…

  8. hooded_paladin says:

    Well, scramble suit should make you think of A Scanner Darkly especially. This is pretty realistic, and so creepy. Now do Pete Burns!

  9. Sirric says:

    I think this is amazing. This is step 2 in the direction of living breathing real life avatars. Just imagine walking down the street with some futuristic glasses, and seeing everyone how they want to be seen. Expand this to full body, get some brand backing. Make an app store with different clothing choices. Thats where I want to see us in 10 years. I like where your heads at. (I’m so punny)

  10. Zig says:

    Anyone else think of The Laughing Man from Ghost In The Shell:Stand Alone Complex?

  11. Phil says:

    Halloween is coming. Build a SnorriCam-like harness for a pico projector and a laptop, and walk around with this stuff projected right on your face. Instant nightmare fuel!

  12. Ptolom says:

    This is fascinating. If they could make it less disturbing, film studios could put famous actors in every movie they make (Every horror movie at least).

  13. anybodysguess says:

    The Very first one looks like he had his nose broken!

  14. Salomon says:

    I wish i can do that

  15. lamer says:

    Better AAM based facial trackers have been around for over 5 years. (GPL too)

    The kid didn’t cite the 10 other works related to this method or post his code for peer review.

    Science fail…

    • Because he was making working software not a scientific paper.
      If coders cited all work in all videos relate to any of their code it would be a mile long. The video libs, the windows driver libs, the 3d algorthms used to project the vertexs from 3d space to 2D, the shading system, probably the compression on the textue formats….it would go on and on.

      We all work on the shoulders of giants as it were.

      oh, and if you think you can make a better one, go ahead.

  16. Cathy Garrett says:

    I would just want it to make my face look like this:

    But not on my own camera… just on everyone else’s camera, and, of course, animated. Can this software do that? =)

  17. t&p says:

    that dude acts like a woman trying on clothing with his face.

  18. At-M says:
  19. “Just imagine walking down the street with some futuristic glasses, and seeing everyone how they want to be seen”

    AR+This tech will be hell cool. Forget fashion, we would have full body avatars that change when we want.
    I’m sure on the one hand people will say this will make us live too much in a fantasy world.

    But, on the other, it would probably make us less materialistic. If anyone or anything can look like anything else for no cost…whats the value of image anymore?

    Wonderfull software either way.

  20. Willrandship says:

    Pretty darn awesome, and I attribute the artifacts to imperfect initial images, mostly. Like, for example, obama’s smile is very wrinkly, and it transferred the wrinkles very heavily onto kyle’s face.

    Glad to see the Apple issue didn’t stop you from hacking some more!

  21. Renee says:

    So has anyone tried this yet? I downloaded the source files but a lot of the file paths seem totally broken for some reason.

  22. Renee says:

    Ok, I found the problem I think. Almost all of the files are available on github but there’s a folder that contains Jason Saragihn Tracker that has some necessary files that aren’t included.

    HAD was mistaken in saying it was included. I requested the files from Saragihn and I’ll see if that was the problem when I get them.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

Join 96,660 other followers