Embedded above is an interesting multitouch demo by [Lahiru]. The goal of the project was to find an easy way to retrofit current LCDs for multitouch. Instead of using infrared or capacitive recognition, it uses a standard webcam mounted overhead. To calibrate, you draw polygon around the desktop screen as the webcam sees it. The camera can identify the location of markers placed on the screen and their color. iDisplay can also recognize hands making the pinch motion and sends these as touch events via TUIO, so it works with existing touch software. It’s written in C++ using OpenCV for image processing with openFrameworks as the application framework.
[via NUI Group]
32 thoughts on “IDisplay, Webcam Multitouch”
1. Surely it should be able to auto-calibrate?
2. Not really ‘touch’ is it? It just tracks the coloured pucks.
3. There’s a pre-written opencv function for tracking colour patches (check out ‘camshiftdemo’).
Looks like about a day’s work!
Looks like this video from MS Research, which has been out for a few years: http://www.youtube.com/watch?v=AtmwQnUlEmc
Wow! Thanks Ryan for that new link! It’s nice to go straight to the originator (plus he explains it.) But too bad about lame interviewer (and the lack of annoying music.)
Nice work! I will make you a functional 3d display for your technology.
johnny lee did some amazing work before with a wii controller. I think johnnys idea is more practical especially since you have the option to click with the press of a button.
However good work though. judos
johnny lee is truly amazing with his creativity!
this is cool its not relly multitouch since u track the pucks but i wanna try this is there a link for a download of the software?
ps another cool thing ive seen is webcam tracks a laser dot and u use it to move pointer and click
Hmm, looks cool to experiment with once the software is released :)
Thanks a lot for the post and thanks a lot for the comments! :)
>1. Surely it should be able to auto-calibrate?
Yeah, auto calibration can be implemented.
>2. Not really ‘touch’ is it? It just tracks the coloured pucks.
It tracks the circular tangible objects.
>3. There’s a pre-written opencv function for tracking colour patches (check out ‘camshiftdemo’).
Well it’s not tracking colours and not using camshift. It is using a custom background subtraction method tht implemented by myself to extract the foreground objects. And after identifying the 2D position it captures the colour info. Since I’m using a normal webcam the displaying image is visible (not like using an IR camera), so removing the background is challenging.
>Looks like about a day’s work!
It’s really great if you can do it within a day. :)
Thanks a lot for the feedback!
iHate this trend….
So why exactly is he tweaking its nipples in that youtube thumbnail?
This looks like something that could benefit the disabled with poor motor skills. A cheap and effective way for them to use a computer. Fine tuned of course.
Great work, none-the-less.
oh btw. I understand what you were going for, but the music was horrible. lol
hehe yeah! should find a better music track. ;)
I saw this post on openframeworks and want to let you know that there I also http://www.openframeworks.info which I use to write articles on some programming techniques.
so…U could exchange the coloured pucks for gloves with colour leds on the tips (activated when U touch the surface)?
Yeah, that’s possible. But I’m trying to allow more natural interactions… using bare hands. :D
http://www.openframeworks.info is really cool! :)
i can see some interesting applications for this.. for instance, mounting a small webcam, like the ones that are installed in most laptops, on the bezel by the touchpad. adjusting the angle via software wouldn’t be too bad, just more precise control. this would make for a more portable approach and put us budget hampered folks a step closer to turning our ancient laptops into multitouch ones :)
Out of curiosity, and I know I am late, would it be possible to have the software calculate a “touch” by watching how much the hand diffuses? Because the ambient light turns the hand into a “blob” to the camera and removes a lot of detail, would it not be possible for it to see the thinning hand/finger/other pointing object and asses when it has made contact with the screen?
Can I use my webcam and my normal LCD screen with this kind of software ? if I can then where I can download this software ?
This website needs more comments haha!! Greetz from Denmark
This is bullshit, copied from MS Research! many users have request the software for try this but no answer, This is called “Open source” right? why are you afraid if you don’t have anything to hide?
Hi Mitchel Reugt,
Thanks for the comment!
I need to clean-up the code base before publishing it as a opensource project. it’s my own research and the code is kinda messy. :)
Anyway since the higher demand. I’ll publish it this month. And please notice that I have nothing to be afraid of. :P
Hope u can publish it ASAP.
Still not published? I would LOVE to use this in my class. I have already implemented a wiimote smartboard, ftir touch screen and home made lcd and doc cam…this would be an amazing add! :) HELP teach our children and POST the software.
You know, If this went open source, it would probably get really clean, really fast. Not to mention be better, have more options ect…
Just an idea. – Yav
I personally know Lahiru and I have seen the effort he has put into develop this prototype. I am 200% sure that it is his own idea and creation. This is just one of his amazing collection of works. So please just appreciate what you see instead of trying to find bulls.
I loved this one , I bookmarked your blog!
Still you haven’t published it :D
Hi, have you published the source and if yes where can I download it.
Never did release that source so I’m guessing that dude had it right?
Please be kind and respectful to help make the comments section excellent. (Comment Policy)