Kinect and TISCH combined for multitouch

[Florian] sent a link to his proof of concept in creating a multitouch display using the Kinect. He’s the one behind the libTISCH multitouch package and that’s what he used to get this working along with the recently released Kinect drivers. He did this on an Ubuntu machine and, although it’s not a turnkey solution he was kind enough to share some rough directions on accomplishing it yourself. Join us after the break for his instructions and some embedded video.

Here’s what [Florian] said when we asked if he could walk us through how to replicate his work:

I’ll try, but it’s really a hack :-) You’ll need

- Ubuntu 10.10
- a Kinect (surprise :-)

- the OpenKinect driver from https://github.com/OpenKinect/openkinect
- go to directory openkinect/c/
- edit lib/CMakeLists.txt so it says “add_library (freenect SHARED ..”
- run “cmake .”
- run “make”

- the TISCH library, 1.1 branch
- svn co https://tisch.svn.sourceforge.net/svnroot/tisch/libtisch-1.1
- compile with something like:
CFLAGS=-I/foobar/openkinect/c/include \
LDFLAGS=”-L/foobar/openkinect/c/lib -lfreenect” \
make install
- go to the libtisch-1.1/build/bin directory
- run “export LD_LIBRARY_PATH=../lib/”
- run “./touchd -Vf”

- if it doesn’t work at first try, edit ~/.tisch.touchd
- in the line with the resolution (line 5), change to something like
“640 480 30 5 127 8 255 0″

- play with the ~/.tisch.touchd settings file (I’m aware it’s quite
horrible, the format is somewhat documented here:
http://sourceforge.net/apps/mediawiki/tisch/index.php?title=Touchd_config ).

Heh, now that I read that, I’m surprised it worked at all :-)

Florian

Your results may vary so please leave any tips you have on getting this up and running in the comments below.

20 thoughts on “Kinect and TISCH combined for multitouch

  1. “I wonder if the kinect could be used to make a 3D scanner…

    Posted at 1:59 pm on Nov 13th, 2010 by Krazeecain”

    The kinect is a 3d scanner, a 4d scanner in fact!

  2. This looks great, but couldn’t openCV be used to create an interface like this?

    Don’t get me wrong, I appreciate anyone sticking it up bill, but to use this interface we still gotta give Micro$oft our bucks for the Kinect.

  3. the point is to create a simple multitouch screen with easy to obtain components( in this case a kinect and various software )

    this is a good hack no matter how much overkill it is.

    REALLY OVERKILL

  4. I gave this a good effort but so far no luck, I was able to get Marvin aka marcan’s code running to pick up the camera by itself. The road bumps I hit with trying to run the instructions above were 1 not quite understanding the part about editing, I just added shared before the other options that were on that line, I was able to get everything to compile it seems so I assume that was right, then like ususal when copying things out of web-pages the quotes get messed up so for the LDFLAGS I had to replace the quotes and of course correct the foobar path. Next one I hit was the libfreenect.so wasn’t in the lib folder of the tisch directory so I copied it in there then everything was able to be built. Upon running the final lines I get this:

    shaun@shaun-laptop:~/Kinect/libtisch-1.1.2/build/bin$ ./touchd -Vf
    touchd – libTISCH 1.1 image processing layer
    (c) 2008 by Florian Echtler
    Caught runtime exception:
    Error: unknown camera type requested.
    Cleaning up.. done. Goodbye.

    I tried modifying the settings file as specified, but so far no luck same message, I will try to reach out to Florian possibly tomorrow if I can’t get this to work out and see if he can offer any advice.

  5. Hmm okay finally got it running I just commented out the section where it’s checking for the camera type against a constant, not sure why that wasn’t coming through as true but creating the KinectImageSource without doing the check worked out okay. Ultimately I also had to run sudo -s before doing the export for LD_LIBRARY_PATH and ultimately for running ./touchd -Vf I believe due to the fact that the underlying libfreenect.so needs access to the USB port. The only other thing to note that I left out earlier was that the LDFLAGS assignment was a bit wrong I had to add /build into mine to get it to a path where it could resolve everything. So far just lots of random spots being detected I’m not sure exactly how this is supposed to work in the first place though so I’ll have to do some more digging around. Giving up for the late late night and will give it all another crack tomorrow, if I don’t get this going by then I may abandon ship and start looking into porting marcan’s work into Java as I’m more comfortable there and then start looking into using inverse kinematics to solve arms/legs (assuming I can pretty easily detect the torso and from there find the extremities to the hands and feet although I’m sure this is no easy task). Either way thanks for putting this out there super excited about all of this, I really feel the system could be better if a room were just rigged with a few camera’s in the corners rather than a couple bunched together in one spot but for what it is this is pretty sweet.

  6. I got everything compiled, but get a runtime exception when running ./touchd -Vf

    “open(/dev/video0): No such file or directory”

  7. Florian mentioned that it’s most likely that lib/simplecv does not contain KinectImageSource.o, which is true in my case. I follow the instructions above to the letter from above, and it compiled fine. I am not seeing any indication during make of it even attempting to compile KinectImageSource. I will investigate further and report back.

  8. Hello, thanks for posting this.

    Did you ever find out what the error message

    “open(/dev/video0): No such file or directory”

    was about?

    I also tried what Shaun talked about and removed all the checks against constants from camera.cc but still no joy.

    I look forward to have TISC working on my box with Kinect.

  9. Anyone figure out the open(/dev/video0): No such file or director issue. I am trying to get this to run, using the directions above, however it seems no matter what I do running touchd shows the “open(/dev/video0): No such file or directory” error. In my case lib/simplecv does contain the KinectImageSource.o file.

  10. Not a noob to Linux but a semi-noob to compiling, I can’t seem to get tisch installed properly with Kinect support.

    1) I compile openkinect from source but I don’t understand:

    “edit lib/CMakeLists.txt so it says “add_library (freenect SHARED ..”

    ; I continue on anyways because glview seems to work fine afterwards.

    2) I also don’t get what the cflags and ldflags are referring to; my openkinect source is in ~/libfreenect, but there is no c/lib folder?

    3) Compiling anyways without those flags and running touchd -Vf just gives me a b&w webcam feed from my laptop’s built-in webcam.

    4) Finally, no ~/.tisch.touchd config file is created automatically; I assume I have to create it myself?

    If anyone can post a step-by-step install of what they did to get this working, I’d be extremely grateful for any help!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s