Speech to Sign Language

According to the World Federation of the Deaf, there are around 70 million people worldwide whose first language is some kind of sign language. In the US, ASL (American Sign Language) speakers number from five hundred thousand to two million. If you go to Google translate, though, there’s no option for sign language.

[Alex Foley] and friends decided to do something about that. They were attending McHack (a hackathon at McGill University) and decided to convert speech into sign language. They thought they were prepared, but it turns out they had to work a few things out on the fly. (Isn’t that always the case?) But in the end, they prevailed, as you can see in the video below.

The heart of the project is a pair of 3D-printed hands. At first, they accidentally printed two left hands. They printed a right hand quickly, but they found out later they were missing one segment which they wound up carving out of wood. Fishing line formed tendons and there were enough servos to require two CPU boards to drive everything.

The speech recognition is from Nuance and if listen closely to the video, you’ll see they are signing “hello” and “goodbye.” We’ll have to take their word for it that the signing is correct and legible.

This reminded us of the sign language gloves from the Hackaday Prize. These hands probably won’t advance the state of the art in prosthetics, but that wasn’t what they were going for.

Thanks to [Butter] for the tip.

20 thoughts on “Speech to Sign Language

    1. I know a Danish guy and a Thai girl, both deaf yet talking via sign.
      He can’t read Thai, she can’t read Danish. An app that converted speech to sign would help both of them, when ncountering people who can’t sign.

        1. sign signals are not universal, italian sign language is different from french sign language and so on. each language sign is the trasposition of her equivalent spoked language.

          1. It’s worse than that: sign language is completely decoupled from the native spoken language. American Sign Language is different to British Sign Language is different to Australian Sign Language.

            You need communication between groups in order to develop a shared language, and it’s only in the past few years that video chat has become possible.

          2. It goes even a bit deeper than that, there are three different sign languages used here in the US. ASL, PSE and SEE. they differ in the syntax being used.

        1. And what if you dont have the money. Or what if the child likes to be deaf. The cochlear is very robot sound coming from it there for not a major fix. Plus they still need hearing aids and such. Its a very painful and invasive surgery. A lot of the deaf community rejects the cochlear and loves being deaf, i personally want signed with deaf people (i live close to Austin, which has one of the largest deaf communities in the US) and they say that they would wait to give their kids a cochlear and if the kids love the deaf community as much as they do great if not, give them a cochlear.

  1. Thing is though that AFAIK sign language combines with facial/mouth expression and it’s not just hands.

    Also would it not be easier to simply create a CGI set of hands and display those on a screen? (for those not aware: many deaf people have trouble learning to read due to the absence of sound so reading can be harder than sign for many.)

  2. Problem is not all countries sign the same, even in the UK BSL the north have different signs than the south for the same words, I am not deaf but am learning so as to communicate with my deaf friends its not easy.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s