This Raspberry Pi 2 with computer vision and two solenoid “fingers” was getting absurdly high scores on a mobile game as of late 2015, but only recently has [Kristian] finished fleshing the project out with detailed documentation.
Developed for a course in image analysis and computer vision, this project wasn’t really about cheating at a mobile game. It wasn’t even about a robotic interface to a smartphone screen; it was a platform for developing and demonstrating the image analysis theory he was learning, and the computer vision portion is no hack job. OpenCV was used as a foundation for accessing the camera, but none of the built-in filters are used. All of the image analysis is implemented from scratch.
The game is a simple. Humans and zombies move downward in two columns. Zombies (green) should get a screen tap but not humans. The Raspberry Pi camera takes pictures of the smartphone’s screen, to which a HSV filter is applied to filter out everything except green objects (zombies). That alone would be enough to get you some basic results, but not nearly good enough to be truly reliable and repeatable. Therefore, after picking out the green objects comes a whole chain of additional filtering. The details of that are covered on [Kristian]’s blog post, but the final report for the project (PDF) is where the real detail is.
If you’re interested mainly in seeing a machine pound out flawless victories, the video below shows everything running smoothly. The pounding sounds make it seem like the screen is taking a lot of abuse, but [Kristian] mentions that’s actually noise from the solenoids and not a product of them battling the touchscreen. This setup can be easily adapted to test out apps on different models of phones — something that has historically cost quite a bit of dough.
If you’re interested in the nitty-gritty details of the reasons and methods used for the computer vision portions, be sure to go through [Kristian]’s github repository where everything about the project lives (including the aforementioned final report.)
Continue reading “Abusing A Cellphone Screen With Solenoids Posts High Score”


The firing assembly in the jet comes from a Nerf Rival Zeus Blaster, which is itself an interesting device. It uses two electric flywheels to launch soft foam balls – much like a pitching machine. With one flywheel running a little faster than the other, the trajectory can be modified. For example, a slight topspin gives the balls a longer and more stable flight path. Of course, foam balls slow down quickly once launched and at high speeds the aircraft can overtake the same projectiles it just fired, but it’s fun all the same.


LaserWeb3 supports different controllers and the machines they might be connected to – whether they are home-made systems, CNC frames equipped with laser diode emitters (such as retrofitted 3D printers), or one of those affordable blue-box 40W Chinese lasers with the proprietary controller replaced by something like a 


![DIY Research Vessel in use, while ROV is busy below. [Source: NYT]](https://hackaday.com/wp-content/uploads/2016/07/openrov-top-end2.jpg?w=250)