Visually impaired people know something the rest of us often overlooks: we actually don’t see with our eyes, but with our brains. For his Hackaday Prize entry, [Ray Lynch] is building a tongue vision system, that will help blind people to see through one of the human brain’s auxiliary ports: the taste buds.
The concept of tongue vision has already been proven. Brainport, a tongue vision system invented by Paul Bach-y-Rita is already available to patients. However, what’s missing is a low-cost open-source hardware alternative that makes the technology available to patients who can’t afford the $10,000 product. [Ray] set out to start this project. The design for his first prototype looks promising already, and all sources will be released as open-source once he can prove that his concept works.
Basically, the system consists of a digital camera that sends image data to a lollipop-like electrode matrix which stimulates a region on the patient’s tongue. In a training phase, the patient learns to interpret the stimulation pattern as the original camera image. For the electrode matrix, [Ray] designed a double-sided PCB with a 16×16 grid of taste-bud-stimulating pixels. Each of this pixels features an exposed via, which acts as the anode, and an exposed ring-shaped pad around the via, which acts as the cathode. The 16 columns of anodes are driven by two MC14555P high-side multiplexer, while the 16 rows of cathode-rings are driven by 74LS156D open-collector multiplexers. Think of it as a LED matrix without the LEDs. The image could be delivered by a Raspberry Pi with attached camera – or by the patient’s smartphone.
If only a few assumptions about a tongue’s actual resistivity and sensitivity prove true, the assembly could indeed produce the desired sensation on a tongue once the first batch of prototype PCBs arrives. Either way, [Ray’s] PCB layout — which is also his first contact with EAGLE — is eye candy, literally. Our tongues are looking forwards to see the prototypes in action!
That’s very interesting concept – I believe babies as there neural networks are been established experience different senses through their taste buds.
Another application could also be neural stimulation via the tongue as n aid to treating epilepsy.
You need to stimulate the vagus nerve to stop a seizure.
Is that the nerve that makes you want to gamble?
I’ve heard of this idea a few years ago and I applaud the open source efforts.
However, I’d be a bit concerned with health when using a pcb orally. Gold plating is a must of course, copper is very toxic, and solder resist mask, fr4 particles, solder and flux residue etc. I’d be reluctant to put my lips on it, let alone my tongue.
Looks like gold plating is planned for future versions.
I’m interested in this kind of HID, it is very intriguing. I would think that everything except the tongue contacts would get coated in silicone or something.
A PCB made from kapton (flex pcb) would be ideal
Copper isn’t “very” toxic. It’s not like cyanide or arsenic that you’ll absorb a lethal dose though a mucous membrane or skin. If significant amount of free copper gets to your stomach it competitively inhibits takeup of iron, leaving you anemic. It’s also an essential nutrient in small amounts. Probably not enough there to harm a healthy adult if they scraped the whole lot off the board and swallowed it… unless they already had anemia, but that’s precluded when you say “healthy”.
It depends on how long you’re exposed to it. I can imagine if you use your tongue as your eyes you’d use it for hours per day for many days.
An interesting thing I saw in some show on Discovery Channel some years ago (before shark week and other silly things) is that infants can identify items by sight after previously having them in their mouths without being able to see them.
Why not in the form of a palate retainer? that way it doesn’t hinder free speech.
how many ‘pixels’ is the tongue capable of actually reading? If the camera relayed contrast information to the pad, could you lick read a page of text?
I was curious as well. I found this article on the two point discrimination density of the regions on the tongue. https://www.ncbi.nlm.nih.gov/pubmed/2817250
“The distribution of minimal two-point discrimination values on the tongue and soft palate was investigated in normal subjects using electric stimulation. It was performed using square pulses of 0.35 ms duration, a rate 10 Hz, a voltage between 3.5 and 7.0 with an average of 4.09 using two silver-silver chloride electrodes (tip diameter, 0.4 mm), and a strain-guage device to control crimping pressure. The spatial threshold was 1.650 +/- 0.433 mm at the tongue tip and 2.650 +/- 0.856 mm and 1.675 +/- 0.269 mm on the anterior margin of each side of the tongue. The soft palate and lingual dorsum were almost completely insensitive. Electric stimulation is a convenient method of measuring two-point limen values.”
I’m thinking military applications, if you had hyper-spectral/NVG/radar wireless to a puck you hold on your tongue
Wouldn’t that be a great application for these cheap-ish low-resolution TOF-cameras to get a 3D-image? Feel obstacles on your tongue…
I wonder how big a Fresnel lens you’d need to focus IR on your tongue and sense it direct……. :-D
I remember tongue-o-vision on the discovery channel. A guy trained with it for a weekend or something and was able to interact with his surroundings. (Walking, glass of water)
Only having high-side on one ring and low-side on another ring…
This will start tasting really bad after a while due to electrolisys.
What this needs to do is apply AC (square wave) to each ring pair using 2×16 tri-state buffers.