Computer handwriting recognition is very cool by itself, and it’s something that we’d like to incorporate into a project. So we went digging for hacker solutions, and along the way came up with an interesting bit of history and some great algorithms. We feel like we’ve got a good start on that front, but we’re stuck on the hardware tablet sensor itself. So in this Ask Hackaday, we’re going to make the case for why you could be using a tablet-like device for capturing user input or doing handwriting recognition, and then we’re going to ask if you know of any good DIY tablet designs to make it work.
The RAND Tablet
The mid-1960s were a heady time for computer research. Fundamental ideas (some of which still seem pretty current) were brewing. At the RAND corporation, they were working on a completely graphical input and programming language. And this meant some way of taking natural user input.
Enter the RAND Tablet. It wasn’t the first tablet, and it surely wasn’t cheap, but it was reportedly among the least expensive around the time. Originally made with wires, it was eventually built with a grid of traces doing capacitive sensing at 100 DPI over 10.24 by 10.24 inches. That’s a full megapixel’s resolution, before the concept of the pixel had even been invented. Not too shabby.
Besides allowing intuitive human annotation of digitized maps and such, the RAND tablet also featured handwriting recognition. And that’s the cool part. The algorithms that it used were very simple — they had to work in real time on an IBM System/360 — but they were elegant and got the job done. [Jack Schaedle] wrote these up, complete with interactive graphics, and you should check it out. Why? Because your average microcontroller today will have as much processing power as the IBM 360, and you could be integrating handwriting recognition into your projects.
How it Works
The idea is to break a character down into a minimal set of features that can distinguish it from others. But even before that can happen, the flood of real-time data from the tablet is run through an exponentially-weighted moving average (EWMA) filter to de-quantize it, and the data points thinned out. The thinning algorithm (don’t record a new point until it’s outside of a given box) is simple and effective.
But it’s the other feature-extraction tricks, like curvature and corner detection, that are really worth your time if you’re interested in making a touchpad. Check out [Jack]’s interactive graphics, and compare the nice results with the simple concepts. In the end, it’s surprising how few bits of information are necessary to distinguish characters from each other. This isn’t “big data”, this is smart engineering.
The RAND tablet’s system would recognize Chinese too — but that’s even easier than English: characters are made up of regularized strokes, always drawn in the same order. This is so much the case that old-school Chinese dictionaries are “alphabetized” according to the series of these strokes. You can look up “horizontal, vertical, left slope, right slope” in the dictionary and discover that it means “tree”. It’s a perfect fit for the recognition system.
Applying this same idea to the roman alphabet required the user to learn a slightly new way of writing. Our characters aren’t composed of regularized brush-strokes. A couple of characters are downright odd. Not coincidentally, we assume, it’s very similar to the alphabet used by the Palm Pilot thirty years later. Xerox would sue Palm for using almost exactly the same technology, so apparently they “discovered” similar tricks at PARC as well.
Hacker Tablets: The Hardware Choices
All of which leads us to ask the question: why aren’t we making tablets for our projects? Digital artists use hard-core digitizing tablets all the time, and they’ve got cool features like pressure sensitivity and so on. They come with amazing DPIs in the thousands, but also with couple-hundred-dollar price tags. Small capacitive-sensing trackpads like the one on your laptop are cheap, but at five-fingers wide they’re not suitable for use as a tablet, but rather as a mouse.
If you require something transparent, capacitive touchscreens are ubiquitous but expensive, and building one from scratch is (nearly) out of the question. Resistive touchscreen overlays are probably the best bet for not much money, and they can be fairly large. Debouncing and cleaning up the data that comes out of them is slightly tricky, but is workable. This is the most plausible non-DIY contender, in my mind, and I’ve got a nice big one to play with. Seven-inch widescreen models are available everywhere for not much money.
Any of these off-the-shelf options should work, but they have drawbacks (expensive, tiny, or feel crappy) and at least the touchscreen interfaces have to be transparent, which this application doesn’t require. With the goal of enabling handwriting recognition and being physically large enough to make using it as a tablet pleasant, we have in mind something approximately the size of a sheet of paper, which can be opaque, but must have decent resolution. Extra points are awarded for capacitive or electrical sensing, because you don’t have to press down so hard as with resistive touch screens. So we went digging for inspiration, and didn’t really come up with much.
First, the RAND tablet itself is a good model. It worked by energizing strips of PCB trace and detecting them through the stylus. Half of the X traces were pulsed high and half low at each step, but done in a way that creates a unique binary code for each position, after ten pulses on the 1024 wires. This was repeated for the Y traces, and the stylus would then know exactly where it was just by reading out the positive/negative pulse pattern. This is very slick, and is very DIY-approachable these days, but requires a wired stylus. If we were recreating this circuit today, we would have to think hard about other ways to send the twenty bits wirelessly.
[Bertho] made a capacitive touchpad that’s driven and decoded by 7400-series logic ICs, and won Dangerous Prototypes’ 7400 Contest in 2011 with it. He then went on to refine his sensor design, but was plagued by all sorts of material problems — glass sticking to molten plastic and bubbles trapped between the sheets. He only built an array of 16 x 16 wires, but interpolation could probably increase the effective resolution enough to be useful. We wonder if the RAND trick of etching the lines into a double-sided PCB wouldn’t work.
Finally, [Micah Elizabeth Scott] has a great two-part series on taking apart commercial USB tablets. This first video provides a lot of USB background information, but gets into the tablet hardware of a cheap tablet (active, inductive!) around twenty minutes in. The second video gets inside a more expensive Wacom tablet, which has an active pen that’s powered by the tablet and works basically like an RFID card. Still, confirming what we know from the RAND tablet, there’s nothing going on that couldn’t be done DIY.
And that’s all we came up with. We vaguely feel like there should be more hacker work done on touch panels and/or touch plates, but we couldn’t find much else. The hardware isn’t impossible and the methods required in firmware, from location detection through to simple handwriting recognition, also look tractable. Why are we not building our own tablets?