Sign language can like any language be difficult to learn if you’re not immersed in it, or at least learning from someone who is fluent. It’s not easy to know when you’re making minor mistakes or missing nuances. It’s a medium with its own unique issues when learning, so if you want to learn and don’t have access to someone who knows the language you might want to reach for the next best thing: a machine that can teach you.
This project comes from three of [Bruce Land]’s senior electrical and computer engineering students, [Alicia], [Raul], and [Kerry], as part of their final design class at Cornell University. Someone who wishes to learn the sign language alphabet slips on a glove outfitted with position sensors for each finger. A computer inside the device shows each letter’s proper sign on a screen, and then checks the sensors from the glove to ensure that the hand is in the proper position. Two letters include making a gesture as well, and the device is able to track this by use of a gyroscope and compass to ensure that the letter has been properly signed. It appears to only cover the alphabet and not a wider vocabulary, but as a proof of concept it is very effective.
The students show that it is entirely possible to learn the alphabet reliably using the machine as a teaching tool. This type of technology could be useful for other applications as well, such as gesture recognition for a human interface device. If you want to see more of these interesting and well-referenced senior design builds we’ve featured quite a few, from polygraph machines to a sonar system for a bicycle.
Would make a great non-verbal 2-way radio interface. It could be ASL to text message or ASL to voice synthesizer. Like where two or more units need to silently send short situational commands to other units or base without being heard.
There are rather too many problems with that for now. To communicate effectively you’d need to use the whole language, not just its alphabet. Sign languages rely on a lot of 3D positional information so you’d need to capture a lot more information than just hand shape and orientation. Translation of that to and from a spoken language is called ‘glossing’ and for now it really requires a full understanding of both languages.
I agree Shannon. However, what I am envisioning is a verbal shorthand method between two or more cohorts familiar with their esoteric shorthand. Like two law enforcement officers desirous of sending silent 10-codes without being overheard verbally by others Ideally full ASL words and concepts would not be used.Simply number codes and or abbreviated words. Similar to US ARMY ranger hand codes. (i.e. go, stop, look over there, move out, caution, etc.) They could be sent electronically over a portable radio transceiver like a walkie talkie.
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSYSdb-zoimaCyn1ClH5ipyJgVOm8NiZtzzk9bHLhV-qKLKf0bt
Text Display or Voice Synth
I believe this will be very helpful, thank you.for a awesome craftmenship.
I think so too !! there are a lot of applications now that help us deaf people communicate with hearing persons, in europe we have djanah ( https://www.djanah.com/be/ ) that basically offers remote interpreting which is cool because we can communicate easily now. hope a big company will invest in this project in order to make people learn sign language easily.