Learning a new language is a great way to exercise the mind and learn about different cultures, and it’s great to have a native speaker around to improve the learning experience. Without one it’s still possible to learn via videos, books, and software though. The task does get much more complicated when trying to learn a language that isn’t spoken, though, like American Sign Language. This project allows users to learn the ASL alphabet with the help of computer vision and some machine learning algorithms.
The build uses a computer vision model in MobileNetV2 which is trained for each sign in the ASL alphabet. A sign is shown to the user on a screen, and the user needs to demonstrate the sign to the computer in order to progress. To do this, OpenCV running on a Raspberry Pi with a PiCamera is used to analyze the frames of the user in real-time. The user is shown pictures of the correct sign, and is rewarded when the correct sign is made.
While this only works for alphabet signs in ASL currently, the team at the University of Glasgow that built this project is planning on expanding it to include other signs as well. We have seen other machines built to teach ASL in the past, like this one which relies on a specialized glove rather than computer vision.
Ack this is pretty cringe-worthy and potentially even harmful to the Deaf community. This “Sign Language AI training tool” is showing INCORRECT hand signs for many classifiers. T, G, P, Q, J (no movement) are all either completely or partially incorrect. in one instance the user signs D with their pinky finger and appears to pass the test. This is a horrendous instance of a non Deaf / non signer bias in technology and a huge reason why there is a need for more native signers in XR tech industry. Please do not create or post Sign Language content without input from the Deaf community. Nothing about us without us.
Nice way to be inclusive and inviting to potential collaborators…
Are you deaf @jrfl?
Maybe the potential collaborators should try to collaborate instead of going off half-cocked. I wouldn’t volunteer to teach people sign language because I don’t know it. That seems like what this person/project have decided to do anyway
Agreed, please stop spreading Incorrect signs!!
Great idea!
Yeah, this isn’t good. It’s not ASL, that’s for sure. It’s made up hand symbols or another signed language.
Sign language or mimic has huge potential for misunderstandings! Be careful!
What is a friendly gesture in one culture, can be an heavy insult in another.
http://isubengal.com/misconceptions-about-sign-language-and-the-deaf-community/
Well, to be fair, this is the case in just about _any_ language. It’s also why we shouldn’t rush to get offended when a mistake is made by a non-native user of the language… and we should always be grateful to those people who are making an effort to learn another language. Learning a language is HARD :-)