Twitch And Blink Your Way Through Typing With This Facial Keyboard

For those that haven’t experienced it, the early days of parenthood are challenging, to say the least. Trying to get anything accomplished with a raging case of sleep deprivation is hard enough, but the little bundle of joy who always seems to need to be in physical contact with you makes doing things with your hands nigh impossible. What’s the new parent to do when it comes time to be gainfully employed?

Finding himself in such a boat, [Fletcher]’s solution was to build a face-activated keyboard to work around his offspring’s needs. Before you ask: no, voice recognition software wouldn’t work, at least according to the sleepy little boss who protests noisy awakenings. The solution instead was to first try OpenCV and the dlib facial recognition library to watch [Fletcher] blinking out Morse code. While that sorta-kinda worked, one’s blinkers can’t long endure such a workout, so he moved on to an easier set of gestures. Mouthing Morse code covers most of the keyboard, while a combination of eye, eyebrow, and other facial twitches and tics cover the rest, with MediaPipe’s Face Mesh doing the heavy-lifting in terms of landmark detection.

The resulting facial keyboard, aptly dubbed “CheekyKeys,” performed well enough for [Fletcher] to use for a skills test during an interview with a Big Tech Company. Imagining the interviewer on the other end watching him convulse his way through the interview was worth the price of admission, and we don’t even care if it was a put-on. Video after the break.

CheekyKeys is pretty cool, doing something with a webcam and Python that we thought would have needed a dedicated AI depth camera to accomplish. But perhaps the real hack here was how [Fletcher] taught himself Morse in fifteen minutes.

17 thoughts on “Twitch And Blink Your Way Through Typing With This Facial Keyboard

  1. Unfortunately some folk who this could most help just lack the muscle control. Which makes me sad since this seems tailor made for quadrapaligics.

    On the other hand imagine this paired with user recognition software so the system knows who is making a specific set of gestures and you literally sign to have specific thigns happen. Like ‘unlok door, turn livingroom lights and TV on’ etc.

    1. Hmm a good point it does seem ideal for them, the few I’ve seen (never in person) do seem like they could probably manage this just fine though, as long as the computer isn’t thrown by the strap they may have to hold their head up comfortably…

    2. Definitely. I built this for my use case (coding), abilities (full facial control most of the time), and restrictions (silent, hands-free). But I can definitely imagine some cases where this might be more broadly useful to others with a little individual customization. If anyone knows first-hand of any specific use cases, I’d love to help out!

    1. Real Morse code, used in landline telegraphy, is a bit different than the now universal international variety. It has some real virtues- it is more “ditty” so is a little faster but interference wasn’t a big deal on landline and international code is, well, more international. Traditional morse, on a sounder, is the kind of thunk-thunk sound you hear in old movies. Only with radio do you get beeps.

      1. I gather at one time some hams would send real Morse code, and build sounders so it would be like telegraph. But this was in the sixties.

        A dash and a dot as a tone is distinctive, a sounder, you’re gearing the difference between a make and break.

  2. Sending morse code was never the problem. Receiving it was. In 1970 I got a telegraph kit (I could switch between morse sounder, buzzer or light) so I could learn the code. It was easy to send, but nobody I knew could send, and that’s what I needed to learn.

    It’s a mistake many make.

    In this case, it’s fine, but if he had to receive morse code, it would take way more than fifteen minutes.

    1. Definitely harder to lear to receive than send. Most recommend not even picking up a key till you are a proficient receiver. I’m well out of practice (at both!) but learning to receive there was a definite shift from picking out characters to just sort of understanding. I learned by LCWO at 25 WPM. You MUST learn it at full speed so you can’t count dots and dashes. Many learn slow and never get over the hump.

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.