Disassembling Mouse Sensors For Tracking Tongues

We just wrapped up the Human Computer Interface challenge in this year’s Hackaday Prize, and with that comes a bevy of interesting new designs for mice and keyboards that push the envelope of what you think should be possible, using components that seem improbable. One of the best examples of this is The Bit, a project from [oneohm]. It’s a computer mouse, that uses a tiny little trackpad in ways you never thought possible. It’s a mouse that fits on your tongue.

The idea behind The Bit was to create an input device for people with limited use of their extremities. It’s a bit like the Eyedriveomatic, the winner from the 2015 Hackaday Prize, but designed entirely to fit on the tip of your tongue.

The first experiments on a tongue-controlled mouse were done with an optical trackpad/navigation button found on Blackberry Phones. Like all mouse sensors these days, these modules are actually tiny, really crappy cameras. [oneohm] picked up a pair of these modules and found they had completely different internal tracking modules, so the experiment turned to a surface tracking module from PixArt Imaging that’s also used as a filament sensor in the Prusa 3D printer. This module was easily connected to a microcontroller, and with careful application of plastics, was imbedded in a pacifier. Yes, it tracks a tongue and turns that into cursor movements. It’s a tongue-tracking mouse, and it works.

This is an awesome project for the Hackaday Prize. Not only does it bring new tech to a human-computer interface, it’s doing it in a way that’s accessible to all.

22 thoughts on “Disassembling Mouse Sensors For Tracking Tongues

  1. Great stuff. Beyond pointer movement, it would interesting to see if it can track and interpret the tongue gestures of speech for text entry, because a) audio speech recog with a pacifier in yer cakehole is gonna be difficult and b) you needn’t use the vocal chords so it could be silent, and c) it would not be sensitive to background noise either.

    1. Tangentially related, but did you see that speech-to-text device a while back which detected silent, sub-vocal movents in your vocal cords when a person merely thinks about words? Apparently those muscles are always weakly connected to the speech center of the brain and you can’t really carry on an internal monologue without activating them. Closest thing to an actual mind-reading machine I’ve seen.

  2. This could actually be pretty useful as another input device if both hands are already busy. That said, I think it would be impossible to simultaneously use it and look dignified. :-) Adults going around with “Binkys” in their mouths? For most people, no. It might make extra work for orthodontists.

  3. I can so easily see cameramen holding their heavy camera steady with both hands and focusing and zooming using one of these things. They don’t have to talk much anyway so keeping something in their mouth could work. Perhaps they could combine it with other functions like blowing in and out to control other things too if they aren’t using the same device as an e-cigarette or something else like a straw.

    1. And have to spit this thing out every time you’ve got to talk to the director or DP, no thanks. Besides, cameras already have zoom and focus in a pretty reasonable and reachable place. But, perhaps this would be a good solution for a disabled camera person who doesn’t have great dexterity.

  4. One might be able to go full wireless and put the whole device in the mouth like a retainer. Then with computer in pocket, OFF TO THE CASINOS!! The potential for discrete machine control and communication is actually pretty large with this.

  5. I think it is a greay idea to use a pacifier to hold the sensor. By using the pacifier “technology” all problems about the sensor getting wet, or the body exposed to chemicals have already been thoroughly tested by the pacifier producers over the past decades. Unfortunately, because it looks like a pacifier, a lot of readers won’t take it seriously and fail to see the true significance of this very interesting an potentially great input device.

    Perhaps the system could be expanded with a small thermistor, sensing the temperature of the operator for additional diagnostics. And perhaps via IR also sense heartbeat and oxygen levels. Imagine this using it with your google glasses (just an example) you can run/exercise, and control your device without talking. The possibilities of hands free control is very very interesting…

  6. Oh man, I really love the ingenuity in this one.

    My only concern is about using this device in a non bone-formed (younger) individuals such as infants or youngsters. This would have a great impact on the development of their yaw bone, mouth-roof bone and overall dental deformities that these pacifiers introduce.

    Other than that, this is a fucking excellent idea! Make it battery powered and invent some sort of movement-to-speach/words gestures and you have a covert eavesdropping device (of course, individual would need to be inside the room to hear and lick the words out). This would make it possible to talk without opening your mouth, just add and earpiece into the system and you are all set!

    1. That particular design of pacifier (Nuk brand or a copy, I think) was designed to not affect the development of the jaw bones in young children. We used that brand for our son who used them until age 3, and he’s never had any orthodontic issues (he’s 24 now).

      This looks to me to be something that could prove very helpful for paraplegics and similarly limited individuals.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.