iDisplay, webcam multitouch

Embedded above is an interesting multitouch demo by [Lahiru]. The goal of the project was to find an easy way to retrofit current LCDs for multitouch. Instead of using infrared or capacitive recognition, it uses a standard webcam mounted overhead. To calibrate, you draw polygon around the desktop screen as the webcam sees it. The camera can identify the location of markers placed on the screen and their color. iDisplay can also recognize hands making the pinch motion and sends these as touch events via TUIO, so it works with existing touch software. It’s written in C++ using OpenCV for image processing with openFrameworks as the application framework.

[via NUI Group]

32 thoughts on “iDisplay, webcam multitouch

  1. Meh.

    1. Surely it should be able to auto-calibrate?
    2. Not really ‘touch’ is it? It just tracks the coloured pucks.
    3. There’s a pre-written opencv function for tracking colour patches (check out ‘camshiftdemo’).

    Looks like about a day’s work!

  2. Wow! Thanks Ryan for that new link! It’s nice to go straight to the originator (plus he explains it.) But too bad about lame interviewer (and the lack of annoying music.)

  3. this is cool its not relly multitouch since u track the pucks but i wanna try this is there a link for a download of the software?

  4. Hi,

    Thanks a lot for the post and thanks a lot for the comments! :)

    ———–

    >1. Surely it should be able to auto-calibrate?

    Yeah, auto calibration can be implemented.

    >2. Not really ‘touch’ is it? It just tracks the coloured pucks.

    It tracks the circular tangible objects.

    >3. There’s a pre-written opencv function for tracking colour patches (check out ‘camshiftdemo’).

    Well it’s not tracking colours and not using camshift. It is using a custom background subtraction method tht implemented by myself to extract the foreground objects. And after identifying the 2D position it captures the colour info. Since I’m using a normal webcam the displaying image is visible (not like using an IR camera), so removing the background is challenging.

    >Looks like about a day’s work!

    It’s really great if you can do it within a day. :)

    Thanks a lot for the feedback!

    -Lahiru.

  5. This looks like something that could benefit the disabled with poor motor skills. A cheap and effective way for them to use a computer. Fine tuned of course.

    Great work, none-the-less.

  6. so…U could exchange the coloured pucks for gloves with colour leds on the tips (activated when U touch the surface)?

  7. i can see some interesting applications for this.. for instance, mounting a small webcam, like the ones that are installed in most laptops, on the bezel by the touchpad. adjusting the angle via software wouldn’t be too bad, just more precise control. this would make for a more portable approach and put us budget hampered folks a step closer to turning our ancient laptops into multitouch ones :)

  8. Out of curiosity, and I know I am late, would it be possible to have the software calculate a “touch” by watching how much the hand diffuses? Because the ambient light turns the hand into a “blob” to the camera and removes a lot of detail, would it not be possible for it to see the thinning hand/finger/other pointing object and asses when it has made contact with the screen?

  9. This is bullshit, copied from MS Research! many users have request the software for try this but no answer, This is called “Open source” right? why are you afraid if you don’t have anything to hide?

  10. Hi Mitchel Reugt,

    Thanks for the comment!

    I need to clean-up the code base before publishing it as a opensource project. it’s my own research and the code is kinda messy. :)

    Anyway since the higher demand. I’ll publish it this month. And please notice that I have nothing to be afraid of. :P

    Thanks!
    -Lahiru

  11. Still not published? I would LOVE to use this in my class. I have already implemented a wiimote smartboard, ftir touch screen and home made lcd and doc cam…this would be an amazing add! :) HELP teach our children and POST the software.

  12. You know, If this went open source, it would probably get really clean, really fast. Not to mention be better, have more options ect…
    Just an idea. – Yav

  13. I personally know Lahiru and I have seen the effort he has put into develop this prototype. I am 200% sure that it is his own idea and creation. This is just one of his amazing collection of works. So please just appreciate what you see instead of trying to find bulls.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s