This is very interesting. What about if 2 of these could be used to triangulate the position and make a table into a touch pad for a computer? It could be integrated to laptops with one on each side in the base.

Posted at 12:00 pm on Nov 14th, 2008 by icefox

could it possible be used to pick up shapes and recognise them as letter. so you could use a wall as a white board

Posted at 12:11 pm on Nov 14th, 2008 by kyle

Doesn’t “TRI”angulation generally require 3 sensors ;) (but I really like that idea icefox, it has me thinking…)

Posted at 12:14 pm on Nov 14th, 2008 by Tuckie

triangulation could allow actual writing, since a w and an m probably sound the same written.

but even then, you’d need enough movement to make enough sound to get translated into mouse movement. I think it should be left for just gestures.

ooh, you could get some multi-touch in there. if the two or 3 mics picked up a sound getting louder on both ends, it could be the user swiping two fingers away from each other, signaling to zoom in…

Posted at 12:36 pm on Nov 14th, 2008 by _matt

Seconded… good thinking icefox

with a triangulated system you would have to calibrate it… unlike the singular mic, but with the right code you could allow for more gestures (eg being able to decipher a V move from a > move…which to a single mic would probably appear the same)

… another project now added to an already large list…DOH!

Posted at 12:44 pm on Nov 14th, 2008 by Tim

After more thinking… if you could get a high enough sample rate, you could even base it on audio delay, as it would take more time for the sound to reach the furthest of the three mics. This would provide another set of data points, and might allow for better resolution.

Posted at 12:52 pm on Nov 14th, 2008 by Tuckie

Triangulation only requires three sensors in 3D space, in 2D you can use only 2 sensors.

Posted at 12:53 pm on Nov 14th, 2008 by Edward Nardella

After more thought… if you could get a high enough sample rate, you could even base it on audio delay, as it would take more time for the sound to reach the furthest of the three mics. This would provide another set of data points, and might allow for a better resolution.

Posted at 12:54 pm on Nov 14th, 2008 by Tuckie

(sorry for the double post, my connection bugged out on me, and didn’t show that it had gone through)

Posted at 1:09 pm on Nov 14th, 2008 by Tuckie

@tuckie: no, triangulation means that you have 2 known points use those to create a triangle with the third unknown position using data gathered at the known positions.

Really, though, I’m not sure this would work since each of the “sensors” is non-directional. You would probably need to use multiple mics with different phase orientations at each position in order to be able to determine an actual angle to the point at which the gesture was made. I’m not very clear on the physics here, but some experimentation would probably show whether this has potential.

A related (but somewhat simpler) project created by my friend and fellow member of DorkbotPDX, Collin Oldham, is his Clapboard: http://homepage.mac.com/coldham/klang/clapboard.html

Posted at 1:37 pm on Nov 14th, 2008 by thomas

Would it be possible to make a table capable of outputting delta X and delta Y data of any given scratch simply by having two sensors at the far top left and far bottom left and measuring the change in amplitude of each. Then using those changes in amplitude + some form of calibration to give something similar to the touch-pad on a laptop?
Of course that assumes that the strength of your scratch is fairly constant, but only for the duration of each individual scratch.

I can’t tell if this is a stupid idea. It’s too early.

Posted at 2:19 pm on Nov 14th, 2008 by Nelson

(Given that x and y are arbitrary units of distance)

Posted at 2:20 pm on Nov 14th, 2008 by Nelson

Trilateration is the term i think you guys are looking for, but that’s for obtaining a position in 3d space using intersecting spheres. For a 2d surface such as a flat wall, only two sensors would be required. To determine distance from the sensors, a time of arrival (phase sensitive detectors), or amplitude method could be used

given sufficient training a machine-learning algorithm would be able to determine the location of a scratch.

Posted at 2:35 pm on Nov 14th, 2008 by Ryan

what’s going on with the comment system, i posted a while ago and it never showed up…i tried to post the same thing and wordpress showed a duplicate comment. ????

Posted at 2:40 pm on Nov 14th, 2008 by macegr

Since most of you seem to be debating my comment about triangulation, let me just throw what I was thinking about when I was thinking of. I do actually agree that 3 may work somewhat better.

Do you know how the police (and other companies) can track cell phone locations? A call tower, or antennae, can determine the range from itself to a point, the phone. This forms a circle of where it could possibly be. A second antennae can form another “range circle” and narrow it down to 2 points in space. A third can be used to determine the exact point the cell phone, rather than 2 possible ones. My old math teacher actually was in the FBI.

Now apply that logic to scratch input, but with much shorter ranges. It could be done with 2, but 3 would be more accurate. 1 however, just seems overcomplicated to me.

Posted at 3:07 pm on Nov 14th, 2008 by icefox

It looks like my earlier comment isn’t going to come back. Anyway, I was pointing out this technology is used in Elo Touch’s acoustic pulse recognition touch panels. I worked with them at elo for a small contract job once.

Basically it’s just two microphones on a sheet of glass. A machine taps the glass in a matrix and the stereo sound profile is recorded, maybe classified into an FFT fingerprint. Then the incoming audio is compared against a database of stored profiles. A normal sound card is enough to get good resolution, though of course Elo’s hardware is optimizing the signal and ADC so you don’t actually have to plug a line in to your computer. All the real work is in the software driver; optimizing the search for known patterns, rejecting noise, making it all fast enough for instant touch response.

You can download whitepapers with more in-depth discussion of the technology on Elo’s website.

Posted at 3:36 pm on Nov 14th, 2008 by macegr

I have to admit, this is a fantastically detailed report for what seems like a weekend experiment. Kudos to the authors, procrastineering, for an innovative little project.

Posted at 3:38 pm on Nov 14th, 2008 by Satiagraha

I feel like you could make a surface that had micromachined groove patterns that were different from different directions, so that each direction (up, down, left, right) produced a unique sound when something scraped across it. Then you could use dead reconing to get an approximate shape of each individual movement, which would make gesture recognition better I wonder if a fresnel lens would be a good place to start? They already have directional grooves in them.
-Taylor

Posted at 4:19 pm on Nov 14th, 2008 by Taylor Alexander

Look back at the recent post regarding the capacitive resistance touchpad using the metallic cloth and 4 points of contact. You could set up 4 scratch sensors and determine x/y location simply by assuming a matrix size they represent and using the strength of the scratch to determine to which side they are closest too.

70% right 30% left = x+20 on a 100/100 grid.

Posted at 4:53 pm on Nov 14th, 2008 by morden

yay im putting microphones in every wall

Posted at 7:18 pm on Nov 14th, 2008 by john

if they get this to work in clothing I’ll really be impressed. if nothing else it will give men the world over an excuse to scratch themselves in public.

Posted at 2:15 am on Nov 15th, 2008 by polymath

I already saw that applied here :
http://www.journaldunet.com/solutions/systemes-reseaux/reportage/08/0208-surfaces-intelligentes-reversys.shtml

It’s in French sorry but the video should be enough.

Posted at 3:46 am on Nov 15th, 2008 by speps

piezo

Posted at 7:53 am on Nov 15th, 2008 by Adam Ziegler

very innovative, i can think of a million different uses for this. How about using a unique gesture to on your car window to open the door?

Posted at 9:03 pm on Nov 15th, 2008 by bountyx

4 sensors would be needed to figure out position. Also a calibration would have to be needed, because each time you do something, even if you tapped the same place, the volume isn’t going to be the same. Maybe tap each corner 3 times. Light, medium and hard.. Just an idea..

Posted at 10:19 pm on Nov 15th, 2008 by EllisGL

I agree. I think for quality of sample and accuracy 4 sensors could be placed on the 4 corners of the surface you are using. I think that as few as 3 could be successfully used, but for a technology like that to be implemented successfully you would need to have >99.999% recognition, or higher. Even 1 false input in 10,000 would cause problems.

That movie in French was good, but I think that you can see there are inherent problems with such things as on surface keyboards, because it would be nearly impossible for the technology to recognize when a button is held, making many hot keys and short cuts unusable.

Posted at 1:30 pm on Nov 16th, 2008 by Mike

@mike – you only need a “hotkey”, i.e. two short taps, followed by your command.

@ellisgl – volume may not be the same, but the sound should always have a relative speed difference among the 4 inputs. Volume “don’t enter into it”, the same thing is in this case – it’s not the volume but the waveform that’s measured/analyzed.

This is a really cool idea… although I must confess I’m thinking along the lines of a security device – could you imagine miking floors, in combination with camera/thermal/etc? You could probably fairly “easily” get the exact location of a person… combine that with a fast turret/airsoft gun combination… w00t! (or real gun, if that happened to be your thing.)

Posted at 10:52 am on Nov 17th, 2008 by srilyk

Fantastic video and project. Well done to all those involved.

Would love to try this out myself. Any idea if the software is available to try?

Thanks to Johnny Lee and his web-site for pointing me in the direction of this project.

Anyone with any details about where to find resources/software for this please post them.

Thanks!

Posted at 8:39 am on Nov 18th, 2008 by Ben

Few things, first off every blog repeating this is saying the same thing “High Pass Filter”

You can easily make this yourself. A First order (6dB per Octave) High Pass filter is simply a Capacitor! Look online to determine how many farads to get the desired filter you wish.

I will be installing one as soon as i can find some sort of software for linux (ubuntu)

I have a complete home audio system connected to one computer running Linux. It would be WONDERFUL to have sound control when we can’t find the Logitech Harmony or the RF keyboard/mouse combo.

Any links to any sort of software?

Posted at 7:23 am on Aug 11th, 2009 by cassetti