Kinect and TISCH combined for multitouch

posted Nov 13th 2010 12:00pm by
filed under: Kinect hacks, multitouch hacks

[Florian] sent a link to his proof of concept in creating a multitouch display using the Kinect. He’s the one behind the libTISCH multitouch package and that’s what he used to get this working along with the recently released Kinect drivers. He did this on an Ubuntu machine and, although it’s not a turnkey solution he was kind enough to share some rough directions on accomplishing it yourself. Join us after the break for his instructions and some embedded video.

Here’s what [Florian] said when we asked if he could walk us through how to replicate his work:

I’ll try, but it’s really a hack :-) You’ll need

- Ubuntu 10.10
- a Kinect (surprise :-)

- the OpenKinect driver from https://github.com/OpenKinect/openkinect
- go to directory openkinect/c/
- edit lib/CMakeLists.txt so it says “add_library (freenect SHARED ..”
- run “cmake .”
- run “make”

- the TISCH library, 1.1 branch
- svn co https://tisch.svn.sourceforge.net/svnroot/tisch/libtisch-1.1
- compile with something like:
CFLAGS=-I/foobar/openkinect/c/include \
LDFLAGS=”-L/foobar/openkinect/c/lib -lfreenect” \
make install
- go to the libtisch-1.1/build/bin directory
- run “export LD_LIBRARY_PATH=../lib/”
- run “./touchd -Vf”

- if it doesn’t work at first try, edit ~/.tisch.touchd
- in the line with the resolution (line 5), change to something like
“640 480 30 5 127 8 255 0″

- play with the ~/.tisch.touchd settings file (I’m aware it’s quite
horrible, the format is somewhat documented here:
http://sourceforge.net/apps/mediawiki/tisch/index.php?title=Touchd_config ).

Heh, now that I read that, I’m surprised it worked at all :-)

Florian

Your results may vary so please leave any tips you have on getting this up and running in the comments below.



20 Responses to Kinect and TISCH combined for multitouch

  • Tom says:
    November 13, 2010 at 12:26 pm

    Haha “Tisch” means table in germen :D

    Reply Report comment
  • Krazeecain says:
    November 13, 2010 at 1:59 pm

    I wonder if the kinect could be used to make a 3D scanner…

    Reply Report comment
  • kislev says:
    November 13, 2010 at 4:18 pm

    someone said “minority report”

    Reply Report comment
  • Paddy says:
    November 13, 2010 at 5:43 pm

    “I wonder if the kinect could be used to make a 3D scanner…

    Posted at 1:59 pm on Nov 13th, 2010 by Krazeecain”

    The kinect is a 3d scanner, a 4d scanner in fact!

    Reply Report comment
  • Cyberteque says:
    November 13, 2010 at 7:27 pm

    This looks great, but couldn’t openCV be used to create an interface like this?

    Don’t get me wrong, I appreciate anyone sticking it up bill, but to use this interface we still gotta give Micro$oft our bucks for the Kinect.

    Reply Report comment
  • baobrien says:
    November 13, 2010 at 8:05 pm

    @Cyberteque I was just thinking that. You could use light pens and a projector/big tv to get a similar effect.

    Reply Report comment
  • baobrien says:
    November 13, 2010 at 8:06 pm

    But, other than that, this is a great use for the newly hacked device.

    Reply Report comment
  • Will says:
    November 13, 2010 at 11:28 pm

    the point is to create a simple multitouch screen with easy to obtain components( in this case a kinect and various software )

    this is a good hack no matter how much overkill it is.

    REALLY OVERKILL

    Reply Report comment
  • gihad says:
    November 14, 2010 at 6:43 pm

    Anyone followed the instructions and got it to work ?

    Reply Report comment
  • Pacman says:
    November 14, 2010 at 8:49 pm

    M$ kinect is a spy tool at M$ disposial, in half the worlds homes…………… hmmmmm…..

    Reply Report comment
  • shaun says:
    November 16, 2010 at 1:09 am

    I gave this a good effort but so far no luck, I was able to get Marvin aka marcan’s code running to pick up the camera by itself. The road bumps I hit with trying to run the instructions above were 1 not quite understanding the part about editing, I just added shared before the other options that were on that line, I was able to get everything to compile it seems so I assume that was right, then like ususal when copying things out of web-pages the quotes get messed up so for the LDFLAGS I had to replace the quotes and of course correct the foobar path. Next one I hit was the libfreenect.so wasn’t in the lib folder of the tisch directory so I copied it in there then everything was able to be built. Upon running the final lines I get this:

    shaun@shaun-laptop:~/Kinect/libtisch-1.1.2/build/bin$ ./touchd -Vf
    touchd – libTISCH 1.1 image processing layer
    (c) 2008 by Florian Echtler
    Caught runtime exception:
    Error: unknown camera type requested.
    Cleaning up.. done. Goodbye.

    I tried modifying the settings file as specified, but so far no luck same message, I will try to reach out to Florian possibly tomorrow if I can’t get this to work out and see if he can offer any advice.

    Reply Report comment
  • shaun says:
    November 16, 2010 at 2:57 am

    Hmm okay finally got it running I just commented out the section where it’s checking for the camera type against a constant, not sure why that wasn’t coming through as true but creating the KinectImageSource without doing the check worked out okay. Ultimately I also had to run sudo -s before doing the export for LD_LIBRARY_PATH and ultimately for running ./touchd -Vf I believe due to the fact that the underlying libfreenect.so needs access to the USB port. The only other thing to note that I left out earlier was that the LDFLAGS assignment was a bit wrong I had to add /build into mine to get it to a path where it could resolve everything. So far just lots of random spots being detected I’m not sure exactly how this is supposed to work in the first place though so I’ll have to do some more digging around. Giving up for the late late night and will give it all another crack tomorrow, if I don’t get this going by then I may abandon ship and start looking into porting marcan’s work into Java as I’m more comfortable there and then start looking into using inverse kinematics to solve arms/legs (assuming I can pretty easily detect the torso and from there find the extremities to the hands and feet although I’m sure this is no easy task). Either way thanks for putting this out there super excited about all of this, I really feel the system could be better if a room were just rigged with a few camera’s in the corners rather than a couple bunched together in one spot but for what it is this is pretty sweet.

    Reply Report comment
  • Nathan says:
    November 17, 2010 at 9:56 am

    I got everything compiled, but get a runtime exception when running ./touchd -Vf

    “open(/dev/video0): No such file or directory”

    Reply Report comment
  • Nathan says:
    November 17, 2010 at 11:01 am

    Florian mentioned that it’s most likely that lib/simplecv does not contain KinectImageSource.o, which is true in my case. I follow the instructions above to the letter from above, and it compiled fine. I am not seeing any indication during make of it even attempting to compile KinectImageSource. I will investigate further and report back.

    Reply Report comment
  • MB says:
    November 17, 2010 at 11:39 pm

    Can Micro$oft still spy on you with the kinect?

    Reply Report comment
  • SomeDude says:
    November 20, 2010 at 8:28 pm

    This might be a stupid question, but how do you connect the actual kinect to your computer?

    Reply Report comment
  • AT says:
    November 22, 2010 at 3:51 am

    Hello, thanks for posting this.

    Did you ever find out what the error message

    “open(/dev/video0): No such file or directory”

    was about?

    I also tried what Shaun talked about and removed all the checks against constants from camera.cc but still no joy.

    I look forward to have TISC working on my box with Kinect.

    Reply Report comment
  • Ryan says:
    November 28, 2010 at 10:48 pm

    Anyone figure out the open(/dev/video0): No such file or director issue. I am trying to get this to run, using the directions above, however it seems no matter what I do running touchd shows the “open(/dev/video0): No such file or directory” error. In my case lib/simplecv does contain the KinectImageSource.o file.

    Reply Report comment
  • Steven Mansour says:
    November 30, 2010 at 7:39 pm

    Not a noob to Linux but a semi-noob to compiling, I can’t seem to get tisch installed properly with Kinect support.

    1) I compile openkinect from source but I don’t understand:

    “edit lib/CMakeLists.txt so it says “add_library (freenect SHARED ..”

    ; I continue on anyways because glview seems to work fine afterwards.

    2) I also don’t get what the cflags and ldflags are referring to; my openkinect source is in ~/libfreenect, but there is no c/lib folder?

    3) Compiling anyways without those flags and running touchd -Vf just gives me a b&w webcam feed from my laptop’s built-in webcam.

    4) Finally, no ~/.tisch.touchd config file is created automatically; I assume I have to create it myself?

    If anyone can post a step-by-step install of what they did to get this working, I’d be extremely grateful for any help!

    Reply Report comment
  • Michael Hoffman says:
    January 7, 2011 at 3:34 pm

    @SomeDude

    the trick to getting it to connect is to get a cable from microsoft that is used with older XBoxs. It’s not that noobish a question because it is a little difficult to find the information or the part.

    http://store.microsoft.com/microsoft/Kinect-Sensor-Power-Supply/product/9A4CFC08

    Still working on getting this to work myself

    Reply Report comment

    • Leave a Reply

    XHTML: You can use these tags: <a href="" title="" rel=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <pre> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

    Hack a Day serves up fresh hacks each day, every day from around the web as well as hacking related news.

    Send us your hacks










         



    Hacks

    Resources