[Chris Harrison] and [Scott E. Hudson] have built a novel system for faking a 3D video chat session. Their implementation separates the image of the chat participant from the background. They then dynamically reposition the video based on the movement of the viewers head. Their using the OpenCV library to do facial recognition (just like the Laughing Man demo). The 3D effect is very similar to what you see in [Johnny Lee]’s Wiimote headtracking. A video of the pseudo 3D chat is embedded below.
[youtube=http://www.youtube.com/watch?v=r8gOgwPgk2g]
The thing that attracts me is not the pseudo 3d effect its the decent effectiveness it managed to remove the background. It would be pretty cool to be able to replace the background (even if it requires a couple of seconds training before a video call) when in ‘sensitive’ areas.
every cheap korean webcam included software can do this.
Any noise in the background makes it useless.
There are some apps that remove the background, others that add snow to the foreground elements. panning the background and foreground is clever. but it’s nothing but a simple filter to those kind of apps. nothing groundbreaking. and rather useless btw.
If there’s a window nearby and a cloud passes, or it someone walks behind the first person, then all image get’s all shitty
Wow, that is cool.
This is a first step. With the proliferation of built in webcams on laptops, netbooks, UMPCs and all kinds of other shit that has Internet, its not too much of a stretch to ask for TWO webcams, on either side. They are pretty cheap!(The webcams)
Theoretically, you can use the above technology, and a well written, Skype, MSN or more professional video conference clone, to take the two images, use the ideas above and get a TRUE three-dimensional picture that is not affected by the things that mr. gabreil talked about (when the background changes).
I would like to see this happen, i think it would be very cool. Also there is the problem, of when (in this first iteration) when u move the camera view, there is black space which looks really unpro.
All in all its a cool idea, that probably could be marketed if you where able to make the software above, remove the bugs and got the appropriate hardware. im thinking two webcams cannabilised into one piece of plastic, with an incorporated USB-Hub i order to have only one usb cable for the end user.
This shit could sell, and also be super cool, and revolusionise the way we video conference.(to say nothing of “other” webcam activities that we may or may not engage in).
wow sorry for the long post boys and girls :S
http://www.engadget.com/2009/01/05/minoru-3d-webcam-ships-this-week-still-looks-freaky?icid=sphere_blogsmith_inpage_engadget
CRAPFUCK! Already done man!!! Im upset, oh well i shoulda known better, but still its not implemented in the way i said but still its there, if expensive and useless in its current form :(
I tried the same thing but using quake instead. It was totally not convincing in real life, even though it looks good on Lee’s videos. Not entirely sure why. Probably something to do with visual cues that you ignore when watching a video.
Also it is pretty trivial to do static background subtraction, even in the presence of noise.
That’s really crazy. I love to see technology being used in creative (new) ways.
Lack of immersion is probably what I experienced on early VR machines: the lag between you moving your head and the display-update is enough to tell your brain that there’s something wrong with the illusion.
Johnny Lee’s Wiimote head tracking is actually a little more realistic given it’s rendered in 3-d depending on the person’s head position and can show things not visible from a straight-on view. This system cannot, and simply rotates/slides two fields being the subject and the background. It’s nifty but certainly nowhere near 3-d and with no visible benefits over standard video chat. If they can make it extrapolate edges and render similarly-colored pixels to extend edges of ‘3-d’ objects, then I would be impressed.
faking 3D? you mean like how CNN did on election night? I suppose if you have enough cameras
how about putting a transparent camera in the middle of the screen so you can actually have eye contact?
I would suggest making the background fill the entire space and to make it so it’s not disproportionate – make the 3D tilting much less dramatic. No one will be ducking and bobbing to the same extreme degrees and this pseudo 3d is distracting rather than more realistic.
awesome.
@will d.
Transparent camera eh, I hear the russians use those to watch paranoid people ;P
I agree the black area that shows is pretty annoying
a simple fix.. although not perfect would be to scale the background larger then the containing window enough to where you don’t see the black edges then you would actually see LESS than normal , BUT you would actually be able to look left and right to see the rest of the background and you wouldn’t get that stupid flat plane floating in space effect on the background and it might seem more like you are actually in the room with the person.
note… I do agree that 2 cameras would make a much better 3D effect, but my advise is just for improving the basic one camera 3d effect depicted here.
They will get even more 3D interaction if they couple the Z-information with the scaling of the front layer (the person). so if a user moves closer to the camera, the openCV tracking sees this and the software can scale the front layer to appear bigger. This will exaggerate the 3D effect significantly with almost no extra effort.
@munden,
The head bobbing and weaving was for demonstrative purposes only. What this really does is tries to make the natural head movements seem more natural to the participants.
I’d love to try this because I think it might make videoconferencing more natural.
ha ha ha
I Love this article! I have been working on Image/Object Recognition as well. I just released a python programmed android app (ported to android) that recognizes objects, people, cars, books, logos, trees, flowers… anything:) It also shows it’s thought process as it “thinks” :)
I’ve put it out as a test for 99 cents on google play.
Here’s the link if you’re interested, there’s also a video of it in action:
https://play.google.com/store/apps/details?id=com.davecote.androideyes
When you are modifying the application within a house router like tp-link, you’ll need a technique to post the program to the tp-link router.
Using factory software package like dd-wrt, producer generally gives a
technique when you are already managing accepted software program.
You use some sort of internet browser attached to your tp-link router in order to upload coming from TFTP server fresh, accepted software right into a regular area.