[Gadget Addict] found out about a contest being held by a shoe seller. Their mobile app has a game very much like Bejeweled. The high scorer each month gets £500. His choices were to be better at the game than everyone else, or to be smarter. He chose the latter by writing a computer vision program to play the game.
There are two distinct parts of a hack like this one. The first is just figuring out a way to programmatically detect the game board and correctly identify each icon on it. This is an iPad game. [Gadget Addict] is mirroring the screen on his laptop, which gives him easy access to the game board and also allows for simulated swipes for automatic play. Above you can see two examples where black pixels may be counted in order to identify the icon. A set of secondary checks differentiates similar entries after the first filtering. The other part of the hack involves writing the algorithms to solve for the best move.
If you liked this one, check out a super-fast Bejeweled solver from several years back. We should also mention that this was just a proof of concept and [GA] never actually entered the contest.
Respect for waiting until the comp was over before going public. Perhaps an even more useful application of such skills is in the various crowd-sourcing games like Phylo, a genome mutation alignment game used to help research into genetic-related disease (http://phylo.cs.mcgill.ca).
This is a good example of KISS – Keep is simple stupid!
Instead of using a complicated opencv algo, he’s just stripping it back to basics and counting pixels.
It looks like the game forces you to wait for all animations to finish before you can make another move, slows things down a lot!
I was expecting to see an arduino with a servo touching slowly and inaccurately at the screen. I’m glad he had the sense to go the software route instead of the ‘oh look I used an arduino’ route.
Whats the best way to mirror the ipad to a computer anyway? Looks pretty smooth in the video.
There’s a vlc client for jailbroken devices, but I imagine mac devtools have something better..
You had to figure out some way to hate on the arduino didnt you? That is just sad.
I like the back but not the disclaimer.
Not even an Arduino in the article and we still get Arduino hate in the comments. Amazing…
in 20 years people will be hating on FPGAs being used to blink an LED.
I hope so. That will mean FPGAs will have become mainstream dev tools. What a refreshing thought.
I seriously doubt FPGAs will ever get anywhere near that popularity in the hobbyist market.