IDisplay, Webcam Multitouch

June 26, 2009 by Eliot 32 Comments

[youtube http://www.youtube.com/watch?v=mlLY0zic7u0%5D

Embedded above is an interesting multitouch demo by [Lahiru]. The goal of the project was to find an easy way to retrofit current LCDs for multitouch. Instead of using infrared or capacitive recognition, it uses a standard webcam mounted overhead. To calibrate, you draw polygon around the desktop screen as the webcam sees it. The camera can identify the location of markers placed on the screen and their color. iDisplay can also recognize hands making the pinch motion and sends these as touch events via TUIO, so it works with existing touch software. It’s written in C++ using OpenCV for image processing with openFrameworks as the application framework.

[via NUI Group]

TUIO Multitouch On IPhone Via Browser Hack

April 22, 2009 by Caleb Kraft 11 Comments

[vimeo=3975324]

When the MSAFluids Remote multitouch App was rejected, a developer found a way to get virtually the same effect using a safari App. They’re using the TUIO protocol to get the desired remote control. This opens up the door to extending Applications through the browser as well. While this probably wouldn’t allow you to utilize the other sensor inputs from your device, like accelerometer data, it will allow for fast prototyping of applications that don’t need that.

IPhone 3.0 Adds Custom Protocol Support For Addons

March 17, 2009 by Eliot 26 Comments

iphone301

In middle of all the adding features that should have been available day-one, Apple announced something really interesting for the hardware hacking community. The new iPhone 3.0 OS will support application communication over bluetooth or through the dock connector using standard or custom protocols. From Engadget’s coverage:

10:19AM “They talk over the dock, and wirelessly over Bluetooth. Things like playing and pausing music, getting artwork — or you can build your own custom protocols.”
10:19AM “Now here’s a class that we think will be really interesting — medical devices.” Scott’s showing off a blood pressure reader that interfaces with the iPhone — wild.
10:18AM “Here’s an example — an FM transmitter. With 3.0, the dev can build a custom app that pairs up with it, and automatically finds the right station and tunes it in.”
10:18AM “With 3.0, we’re going to enable accessory developers to build custom apps that talk directly to that hardware.”

No solid connection specification has been published yet. We’re excited about the prospect of developing our own accessory hardware, but we wonder what sort of hoops you’ll have to jump through. Apple doesn’t have the best track record when it comes to approvals. Just this week they denied MSA Remote client App Store entry; it’s a multitouch client that uses the standard TUIO protocol. Prepare for similar roadblocks in the future.

[via adafruit]

Nintendo DS OSC Support

December 6, 2008 by Eliot 22 Comments

dsmi

OpenSound Control protocol is an emerging standard for communication between musical programs. It’s meant to replace MIDI. The DSMI, DS Music Interface, team has just added support for OSC. You can now use your DS as generic OSC music controller over WiFi. OSC has TCP/IP support built in, so there is no need to run a host sever to talk to DSMI like you did when they only supported MIDI. We’ve seen OSC used in other projects like the monome. It’s also the basis for the multitouch communication protocol TUIO.

[via CDM]

Tbeta, Cross-platform Multitouch

November 28, 2008 by Eliot 5 Comments

[vimeo=http://vimeo.com/2034557]

tbeta is a new tool developed by the NUI Group Community. tbeta acts as an image processing layer to take in image data and output tracking data for multitouch applications. Whether FTIR or DI, scratch built multitouch systems generate IR video streams that need to be processed to find fingertips. tbeta can take this or any arbitrary video stream and run it through a series of filters to generate the touch data. This data is sent as OSC TUIO, a standard protocol for touch events. Along with the camera and input switcher, tbeta also aids in system calibration. I works on Windows, OSX, and Linux. Have a look at the getting started guide for a better idea of how it works.

[via CDM]