Computers haven’t done much for the quality of our already poor handwriting. However, a man paralyzed by an accident can now feed input into a computer by simply thinking about handwriting, thanks to work by Stanford University researchers. Compared to more cumbersome systems based on eye motion or breath, the handwriting technique enables entry at up to 90 characters a minute.
Currently, the feat requires a lab’s worth of equipment, but it could be made practical for everyday use with some additional work and — hopefully — less invasive sensors. In particular, the sensor used two microelectrode arrays in the precentral gyrus portion of the brain. When the subject thinks about writing, recognizable patterns appear in the collected data. The rest is just math and classification using a neural network.
If you want to try your hand at processing this kind of data and don’t have a set of electrodes to implant, you can download nearly eleven hours of data already recorded. The code is out there, too. What we’d really like to see is some easier way to grab the data to start with. That could be a real game-changer.
More traditional input methods using your mouth have been around for a long time. We’ve also looked at work that involves moving your head.
Shouldn’t the title be “Handwriting via thought control?”
TBF, right now I’m controlling a small fraction of your thoughts with my typed text. Adapting this scheme to a handwritten interface should be trivial under the right circumstances.
No, I’m pretty sure they’re typing.
Although I couldn’t find the input numbers from the “more cumbersome methods”. Which’d be nice if we’re making a comparison.
The whole trick of this research is that a handwritten character involves a bunch of distinct motor activations. Basically, you encode the character in a way that’s relatively easy to pick up in motor cortex, instead of forcing the participant to navigate to a specific spot on a screen using cursor control. You don’t actually end up with handwritten characters, that’s just what the participant is trying to do internally. It’s a clever trick for speeding up basic input.
If it wasn’t for the electrodes in your head this would be a great replacement for a keyboard. Also I wonder how this would work for Chinese, or Sign Language?
If one thinks in Russian, one can launch a few missiles.
Spitballing: Chinese would be trickier b/c of the raw number of characters to match against. I’m not sure how the language’s structure will help / hurt. (Lots of words are two characters, which should give the network some context to work with, like with the English frequency of following letters.)
Writing Chinese is also a lot more physical — focus on the strokes kinda replaces our focus on individual letters. Maybe that helps give cleaner signal? I’d expect the same out of sign language.
Love to see the research go in this direction!
No electrodes in my head, unless necessary, please.
More characters to match against, but equally more information per character. And Chinese handwriting has specifically defined stroke orders and directions. It’d probably work great!
Yo, dawg. I hear you like neural nets. So I put a neural net in your neural net so that you can, er, type.
I wonder how hard to train it to pick up on thinking of typing instead of writing? I wear out erasers when handwriting anything because I only know how to spell by typing.
Not mentioning Elon??
Do you know something that we don’t?