Masks are all well and good when it comes to reducing the spread of deadly pathogens, but they can make it harder to understand people when they speak. They also make lipreading impossible. [Kevin Lewis] set about building something to help.
The system consists of a small screen that can be worn on the chest or other part of the body, and a lapel microphone to record the wearer’s speech. Using the Deepgram AI speech recognition API running on a Raspberry Pi Zero W, the system decodes the speech and displays it on the Hyperpixel screen.
The API is quite capable, and can be set to only respond to the wearer’s voice, or in a group mode, display speech from multiple people in the area, displaying other voices in another colour. There’s also a translation feature using the iTranslateApp API as well.
It’s a neat tool that could be of great use in conferences or in situations where a quick simple machine translation could majorly ease communication. Video after the break.
Masks making it hard to understand people either audibly or because you rely on lip reading? Just got this hacked together – it displays my speech in real time with @DeepgramAI! pic.twitter.com/lPu4CZboIk
— Kevin Lewis (he/him) (@_phzn) January 4, 2022
18 thoughts on “Display Your Speech In Realtime To Help Lipreaders In The Mask Era”
Live TV presentations could benefit from this by having subtitles and closed captions without manual labor.
not with tha advertised accuracy of just 90%
What do you mean, us deaf people don’t want youtube automatic captions on TV when we can have accurate human-written subtitles ?
Is this a ‘like’ retweet or a deepgram ad or where is the project?
And where is that video after the break?
The video is in the linked tweet (also, first link of the post) – https://twitter.com/_phzn/status/1478504862170161152
Sorry, must have been blocked on my browser.
And how well will that work in a noisy environment? A mic in the mask should help with that. Maybe combined with a throat mic to confirm the source for really loud environments.
Masks don’t protect me from you, they protect everyone else from your inconsiderate, possibly-infected ass. Please go die of COVID as quickly as possible and leave this world, you selfish fuck.
How about making the mask a bit bigger and putting a camera inside with fog proof lens to video your mouth and show it outside the mask? Lip-reading will be very possible with minimal distortion.
I think it’s near impossible to focus on the whole mouth at that kind of distance. My first thought was a clear mask with filters over the cheeks where they don’t block the view. Prototypes have been featured in articles almost since this mess started but I haven’t seen any for sale.
Will we cope through hopeful acquired immunity, or are you shooting for a Darwinian Wave? Or are you in the straight up cuckoo Corona is Fake camp?
I’m glad no one infringed your freedom to spread disease.
be careful tho, vi hear you can get viruses from computers now too.
MFW you realize that even the CDC and WHO call your cloth mask useless:
One day people like you will look back and realize you’ve been hoodwinked. Instead of admitting it, you’ll move the goalposts and rationalize your poor behavior with this phrase:
“I was just doing what I was told.”
nice, but i need assistent open source and offline, no sending trought internet my voice and data
this working in Pilish language too?
This is cool! Then add a translator for a second langue for us who can’t speak the native language. Thanks for sharing!
Is this an open project? I’d like to build a prototype for my spouse’s school.
It looks like the library is proprietary.
Please be kind and respectful to help make the comments section excellent. (Comment Policy)