At this year’s Pycon [Jason Huggins] gave a talk about his Angry Birds playing robot. He built a delta robot which includes a pen actuator for controlling a capacitive touch screen. The video after the break starts with a demo of the bot beating a level of Angry Birds on the iPad.
The idea behind the build is that robots like this could be used for app testing. I this case [Jason] has tweaked the servo commands manually to achieve the results. But during the talk he does demonstrate some machine vision to analyze and win a game of tic-tac-toe.
We do enjoy seeing the robot, but we’re not sold on the thought that testing will use robots. Perhaps there is a niche need for this type of thing, but we assume the majority of automated testing can be done in the emulator for the device on which you are developing. What we really want to know is how the capacitive stylus works. We didn’t catch him talking about it at all. We want a reliable, yet simple way to electronically trigger touchscreen inputs (along the lines of this project). If you know how [Jason’s] stylus is working please share your thoughts in the comments section.
Continue reading “Shakey robot plays Angry Birds”
[Rob Morris] has been hard at working improving his guitar augmentation techniques. Here he’s demonstrating the use of an iPhone to control the effects while he plays. This builds on the work he shared a few years ago where he strapped a Wii remote to the body of his ax.
Just like the Wii remote, the iPhone includes an accelerometer. As you would expect the best parts of the older hack made it into this one, but the inclusion of the touch screen adds a lot more. In the clip after the break he starts by showing off the screen controlling a whammy bar functionality. But we really love the octave offset feature that comes next. This kind of sound manipulation simply can’t be done using a purely physical method (like the whammy bar can). But he’s not done yet. The demo finishes with a Theremin feature. You’ll notice he plucks a string but no sound comes out until he starts touching the screen. This turns it into an entirely different type of instrument.
The only info we have about putting this together is the list of packages he’s using: TouchOSC, Max/Msp, and GuitarRig
Continue reading “iPhone wielding guitar adds tip of your finger or tip the instrument control”
We love capacitive touch screens. They’re much more robust than resistive touch screens and if the UI is programmed well they produce a great user experience. But getting your electronics project to interact with one is a bit tough. [RobB] has been experimenting in that area, and managed to build a simple touchscreen actuator for microcontroller use.
In the video after the break you can see his proof of concept. He’s using an Arduino to enter the number 2 on an
Android iOS calculator app once every second. It doesn’t take much to pull off this trick, [RopB] just taped a piece of tin foil to the screen and connected it to the Arduino with a jumper wire. The pin is left floating until a screen tap is needed, at which point it is pulled to ground.
A custom app operating at slow speeds could use this as an input technique. Two pieces of foil (one acting as clock, the other data) would provide a rudimentary serial transfer system.
Continue reading “Reaching out to a touch screen with a microcontroller”
This rig will take the letters you write on the touchpad using a stylus and turn them into digital characters. The system is very fast and displays near-perfect recognition. This is all thanks to a large data set that was gathered through machine learning.
The ATmega644 that powers the system just doesn’t have the speed and horsepower necessary to reliably recognize handwriting on its own. But provide it with a dataset to compare against and you’re in business. [Justin] and [Stephen] designed a neural network algorithm that took a large volume of character handwriting samples, and boiled them down into a set of correlations that can be referenced when encountering a new entry. This set is about 88 kilobytes, too much to store in the microprocessor, but easy to reference from an external flash memory device.
There’s plenty of gritty details in the write up linked above, but you may want to start with the video overview found after the break.
Continue reading “Machine learning lets micro decode your handwriting”
[Andy Brown] has been working on a series of tutorials revolving around the STM32 processor family. He’s using the STM32plus development board, with an STM32F1 ARM Cortex M3 processor to drive a couple of different full color graphic LCD screens. His latest installment shows how to read from the touch screen included with both displays.
After the break we’ve embedded the video from which this screenshot was taken. As an example, [Andy] has programmed a painting program to show off what the touchscreen overlay is capable of. It starts off with the calibration routine we’re all familiar with, then drops to this screen with a virtual control panel and blank canvas.
This hardware uses the Texas Instruments ADS7843 controller, which [Andy] says is extremely common and that several other manufacturers use the same communications protocols. He discusses how to communicate with the controller, and how to incorporate the data into your program. Included is an open source library which you can use in your own projects.
Continue reading “Using a touch screen with an STM32 microcontroller”
Forget Microsoft Surface, what do you think about having a 32-inch Android-powered touchscreen display in your living room? That possibility might not be too far off, thanks to the engineers over at SKR Technology in Japan.
Primarily a company that designs and builds digital signage, they were approached by several customers who wanted a large screen device that had multi-touch functionality similar to a smartphone. Since they frequently work with Windows, they tried building a solution around Windows 7, but it just didn’t function as smoothly as they would like. Instead they turned towards Android, but were disappointed to find out that none of their suppliers supported the OS.
Instead of scrapping the project, they build their own interface that allows an Android-powered device to interact with multi-touch displays. As you can see in the video embedded below the display works quite well, mirroring everything on the Android device’s screen.
While the product is not yet available commercially, we should see it come to market later this year. We hope to see an open source version sometime in the future as well, even if we can’t quite afford a 32” touch panel display.
Continue reading “Running Android on large touch screen displays”
[Jane] wrote in to let us know about the touch-based synthesizer she and her classmates just built. They call it the ToneMatrix Touch, as it was inspired by a flash application called ToneMatrix. We’re familiar with that application as it’s been the inspiration for other physical builds as well.
A resistive touch screen in the surface glass of the device provides the ability to interact by tapping the cells you wish to turn on or off. Below the glass is a grid of LEDs which represent sound bits in the looping synthesizer track. Fifteen shift registers drive the LED matrix, with the entire system controlled by an ATmega644 microcontroller. Although the control scheme is very straight forward, the jumper wires used to connect the matrix to the shift registers make for a ratsnest of wireporn that has been hidden away inside the case. Check out the demonstration video after the break to see what this looks like and sounds like when in use.
Continue reading “Touch-based synthesizer is a wiring nightmare”