TUIO multitouch on iPhone via browser hack

When the MSAFluids Remote multitouch App was rejected, a developer found a way to get virtually the same effect using a safari App. They’re using the TUIO protocol to get the desired remote control. This opens up the door to extending Applications through the browser as well. While this probably wouldn’t allow you to utilize the other sensor inputs from your device, like accelerometer data, it will allow for fast prototyping of applications that don’t need that.


  1. jproach says:

    If you like the visuals, try Plasma Pong:

  2. blizzarddemon says:

    could this work on the itouch too?

  3. Peter Kirn says:

    Yep, I’m using it on an iPod touch!

  4. cptfalcon says:

    how are mutltiple touches handled in safari? does each touch get a special event-id so that you can distinguish which finger is moving on a move update? does this require a specific safari js interpreter, or could it be used any multitouch browser?

  5. Gadgeteer says:

    This is NOT running in Safari. He is using a downloaded iPhone app called OSCremote which happens to be configured using HTML. More info here:

  6. cptfalcon says:

    thanks for the clarification, apparently i miss-understood the bit about oscremote

  7. michu says:

    lcd4linux (bleeding edge) does support the vnc protocol – also buttons are supported. so you may use lcd4linux and a vnc viewer on your iphone to remote control something (eg. music player)

  8. Peter Kirn says:

    gadgeteer, that’s incorrect. He was inspired by the fact that OSCemote uses essentially an embedded Web interface. It is absolutely possible to run an application in Safari, which is what he’s doing here. It is a Web app. Arguably, so is OSCemote, but that’s not what he’s using.

  9. Andrew Turley says:

    @cptfalcon, the described program uses Safari, not OSCemote, like Peter says. The touch events have a list of touches, and each touch has a property called “identifier” that is unique for that touch within the lifetime of the touch. But the identifiers (which are just integers) can be reused. When a touch ends, a touchend event is sent out (and received by listeners), and you can get the touches from that event to determine when a given touch has ended. That way you can know that the current touch #7 is different from the touch #7 you were following earlier, because there was a touchend event with touch #7 in between the two.

    @gadgeteer, sorry about the confusion. I think I wrote that at around 2:00 AM, so I may not have been clear.

    @peter, I would call OSCemote “a platform for specialized web apps” rather than an actual web app. I mean, you can open any web page from within the app.

  10. Mitch says:

    This is why apple sucks. They have an absolutley amazing piece of hardware, and instead of opening it up so people can do revolutionary things, they have to “approve” applications and decide to block things that don;t fit with apple. I think Wired sumed it up best with the following quote… “your iPhone is a pathetic piece of DRM-riddled crippleware”…

  11. cptfalcon says:

    thanks andrew, i had no idea the web app stuff for the itouch had support for multitouch, that is pretty neat… might have to see what neat tricks i could do with a multitouch page :)

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

Join 96,725 other followers