Perspective tracking with only a web cam

posted Dec 11th 2009 8:31am by
filed under: news

[Johhny chung Lee], eat your heart out. Check out what these guys are doing with face tracking and immersive 3d as their final project in class. They’re using a singe camera and an FPGA to produce the demo you see in the video. Facial tracking is done by skin color, so that might have some issues in some environments, but being able to have perspective shift with you, without rigging up some more hardware is fantastic.

We realize that this is completely different that what [Johnny] is doing. We love [Johnny]‘s work and think it is ground breaking to be able to pull this stuff off with a cheap game controller. We just couldn’t help but draw the parallel from the end result.

[thanks Bruce]



18 Responses to Perspective tracking with only a web cam

  • andrew says:
    December 11, 2009 at 8:57 am

    OpenCV has face tracking in the example code section. I wonder why they didn’t use that.

    Reply Report comment
  • djrussell says:
    December 11, 2009 at 9:00 am

    1. the guy in the back needs to put some pants on. too much leg.

    2. ever play a video game and really get into enough to be leaning and moving around as if you could look around objects? this could actually make it work and would be a really cool in game effect.

    Reply Report comment
  • toodlestech says:
    December 11, 2009 at 10:12 am

    My webcam came stock with face tracking drivers and physically moves if I start to go out of range of its sight… for $40.

    Reply Report comment
  • pascal says:
    December 11, 2009 at 10:25 am

    @andrew it looks like the prime reason for this project is the FPGA, they seem to process the webcam image and generate the output image using only that prototype board. On a GPU it would require 50 lines of OpenGL/CV-Code, but that’s really not the point.

    Reply Report comment
  • pascal says:
    December 11, 2009 at 10:27 am

    (also I think the “without rigging up some hardware”-Part is hilarious, since their solution *is* pure hardware^^)

    Reply Report comment
  • Chris says:
    December 11, 2009 at 10:29 am

    > OpenCV has face tracking in the example code section. I wonder why they didn’t use that.

    fun? or because it was for a class that probably wasnt about using OpenCV example code.

    >My webcam came stock with face tracking drivers and physically moves if I start to go out of range of its sight… for $40.

    not nearly as cool as building it yourself on FPGA though huh? Since when is hack a day full of old curmudgeons that think its better to just buy the completed project instead of doing it yourself?

    Reply Report comment
  • Chuck says:
    December 11, 2009 at 10:53 am

    @andrew We did consider using OpenCV’s face tracking function for our project. But we didn’t know how to use OpenCV library with the Nios II soft processor that we implemented on the FPGA.

    Reply Report comment
  • yoyoyo says:
    December 11, 2009 at 1:02 pm

    artificial mirror

    Reply Report comment
  • wally says:
    December 11, 2009 at 2:44 pm

    @andrew: I did exactly that for part of my dissertation at Essex Uni. It didn’t take much to implement it …

    Reply Report comment
  • Snowy says:
    December 11, 2009 at 8:13 pm

    Either that guy was mumbling or my hearing is going. With all the background noise and him not looking at the camera when talking/mumbling, I couldn’t understand a word he said.

    Reply Report comment
  • Kaepora says:
    December 11, 2009 at 9:24 pm

    I can imagine great and fun immersive video games where each individual limb can be tracked independently. This is a great start.

    (I saw this guy in the lab the other day…leaning around and looking at a bunch of squares…..now I know what he was doing.)

    Reply Report comment
  • wally says:
    December 12, 2009 at 3:12 am

    @andrew: That said, having looked over this guy’s videos a second time, they look much smoother than I managed. Kudos to them :)

    Reply Report comment
  • sean says:
    December 12, 2009 at 9:27 am

    Okay, I don’t mean to hate, but….
    what he’s doing is so completely basic that it isn’t complicated at all. If you download processing, one of the examples associated with web cam programming is almost this exact same thing…
    All this guy is doing is putting a dot in the center of the brightest object detected by the camera. Sure, this COULD be used as perspective, but technically perspective has to do with the space in which you can see. All this is doing is tracking your head’s placement… not in which direction you are looking. True perspective software would focus on your eyes.
    Now I agree that this opens up a lot of possibilities for games and etc., but I don’t think that what this guy has done should be a huge deal.

    Otherwise, good find.

    Reply Report comment
  • walt says:
    December 13, 2009 at 3:43 pm

    that dude in the background needs to put on some shoes and socks. what is with all the youtube videos of people in their bare feet? GROSS!

    Reply Report comment
  • anon says:
    December 21, 2009 at 3:34 am

    it is human feet. get over it.

    Reply Report comment
  • Yaniv says:
    December 22, 2009 at 10:48 am

    A similar project back from 2008, using openCV and Johnny Lee’s software:

    Reply Report comment
  • Ahmed Sokar says:
    March 18, 2010 at 8:58 am

    Nice work

    May you guide me to run OpenCv on NiosII

    Reply Report comment

    • Leave a Reply

    XHTML: You can use these tags: <a href="" title="" rel=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <pre> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

    Hack a Day serves up fresh hacks each day, every day from around the web as well as hacking related news.

    Send us your hacks










         



    Hacks

    Resources