Perspective tracking with only a web cam

[Johhny chung Lee], eat your heart out. Check out what these guys are doing with face tracking and immersive 3d as their final project in class. They’re using a singe camera and an FPGA to produce the demo you see in the video. Facial tracking is done by skin color, so that might have some issues in some environments, but being able to have perspective shift with you, without rigging up some more hardware is fantastic.

We realize that this is completely different that what [Johnny] is doing. We love [Johnny]‘s work and think it is ground breaking to be able to pull this stuff off with a cheap game controller. We just couldn’t help but draw the parallel from the end result.

[thanks Bruce]

19 thoughts on “Perspective tracking with only a web cam

  1. 1. the guy in the back needs to put some pants on. too much leg.

    2. ever play a video game and really get into enough to be leaning and moving around as if you could look around objects? this could actually make it work and would be a really cool in game effect.

  2. @andrew it looks like the prime reason for this project is the FPGA, they seem to process the webcam image and generate the output image using only that prototype board. On a GPU it would require 50 lines of OpenGL/CV-Code, but that’s really not the point.

  3. > OpenCV has face tracking in the example code section. I wonder why they didn’t use that.

    fun? or because it was for a class that probably wasnt about using OpenCV example code.

    >My webcam came stock with face tracking drivers and physically moves if I start to go out of range of its sight… for $40.

    not nearly as cool as building it yourself on FPGA though huh? Since when is hack a day full of old curmudgeons that think its better to just buy the completed project instead of doing it yourself?

  4. @andrew We did consider using OpenCV’s face tracking function for our project. But we didn’t know how to use OpenCV library with the Nios II soft processor that we implemented on the FPGA.

  5. Either that guy was mumbling or my hearing is going. With all the background noise and him not looking at the camera when talking/mumbling, I couldn’t understand a word he said.

  6. I can imagine great and fun immersive video games where each individual limb can be tracked independently. This is a great start.

    (I saw this guy in the lab the other day…leaning around and looking at a bunch of squares…..now I know what he was doing.)

  7. Okay, I don’t mean to hate, but….
    what he’s doing is so completely basic that it isn’t complicated at all. If you download processing, one of the examples associated with web cam programming is almost this exact same thing…
    All this guy is doing is putting a dot in the center of the brightest object detected by the camera. Sure, this COULD be used as perspective, but technically perspective has to do with the space in which you can see. All this is doing is tracking your head’s placement… not in which direction you are looking. True perspective software would focus on your eyes.
    Now I agree that this opens up a lot of possibilities for games and etc., but I don’t think that what this guy has done should be a huge deal.

    Otherwise, good find.

  8. that dude in the background needs to put on some shoes and socks. what is with all the youtube videos of people in their bare feet? GROSS!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s