IDisplay, Webcam Multitouch


Embedded above is an interesting multitouch demo by [Lahiru]. The goal of the project was to find an easy way to retrofit current LCDs for multitouch. Instead of using infrared or capacitive recognition, it uses a standard webcam mounted overhead. To calibrate, you draw polygon around the desktop screen as the webcam sees it. The camera can identify the location of markers placed on the screen and their color. iDisplay can also recognize hands making the pinch motion and sends these as touch events via TUIO, so it works with existing touch software. It’s written in C++ using OpenCV for image processing with openFrameworks as the application framework.

[via NUI Group]

32 thoughts on “IDisplay, Webcam Multitouch

  1. Meh.

    1. Surely it should be able to auto-calibrate?
    2. Not really ‘touch’ is it? It just tracks the coloured pucks.
    3. There’s a pre-written opencv function for tracking colour patches (check out ‘camshiftdemo’).

    Looks like about a day’s work!

  2. Hi,

    Thanks a lot for the post and thanks a lot for the comments! :)


    >1. Surely it should be able to auto-calibrate?

    Yeah, auto calibration can be implemented.

    >2. Not really ‘touch’ is it? It just tracks the coloured pucks.

    It tracks the circular tangible objects.

    >3. There’s a pre-written opencv function for tracking colour patches (check out ‘camshiftdemo’).

    Well it’s not tracking colours and not using camshift. It is using a custom background subtraction method tht implemented by myself to extract the foreground objects. And after identifying the 2D position it captures the colour info. Since I’m using a normal webcam the displaying image is visible (not like using an IR camera), so removing the background is challenging.

    >Looks like about a day’s work!

    It’s really great if you can do it within a day. :)

    Thanks a lot for the feedback!


  3. This looks like something that could benefit the disabled with poor motor skills. A cheap and effective way for them to use a computer. Fine tuned of course.

    Great work, none-the-less.

  4. i can see some interesting applications for this.. for instance, mounting a small webcam, like the ones that are installed in most laptops, on the bezel by the touchpad. adjusting the angle via software wouldn’t be too bad, just more precise control. this would make for a more portable approach and put us budget hampered folks a step closer to turning our ancient laptops into multitouch ones :)

  5. Out of curiosity, and I know I am late, would it be possible to have the software calculate a “touch” by watching how much the hand diffuses? Because the ambient light turns the hand into a “blob” to the camera and removes a lot of detail, would it not be possible for it to see the thinning hand/finger/other pointing object and asses when it has made contact with the screen?

  6. This is bullshit, copied from MS Research! many users have request the software for try this but no answer, This is called “Open source” right? why are you afraid if you don’t have anything to hide?

  7. Hi Mitchel Reugt,

    Thanks for the comment!

    I need to clean-up the code base before publishing it as a opensource project. it’s my own research and the code is kinda messy. :)

    Anyway since the higher demand. I’ll publish it this month. And please notice that I have nothing to be afraid of. :P


  8. Still not published? I would LOVE to use this in my class. I have already implemented a wiimote smartboard, ftir touch screen and home made lcd and doc cam…this would be an amazing add! :) HELP teach our children and POST the software.

  9. I personally know Lahiru and I have seen the effort he has put into develop this prototype. I am 200% sure that it is his own idea and creation. This is just one of his amazing collection of works. So please just appreciate what you see instead of trying to find bulls.

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.