Check out this brainy bot with [Jari] whipped up to dominate the Bookworm Deluxe scoreboard. The bot runs on top of a win32 machine, pulling screenshots to see the game board and simulating mouse clicks to play. The video after the jump shows that it plays like a champ, but it took some doing to get this far and [Jari] took the time to share all of the development details.
The hardest part of writing these types of bots is recognizing the game pieces. Check out all of the animation that’s going on in the still shot above… a lot of the tiles are obscured, there are different colors, and the tiles themselves shift as the bot spells and submits each word.
After some trial and error [Jari] settled on an image pre-processor which multiplies pixel values by themselves four times, then looks at each pixel with a 1/6 threshold to produce a black and white face for each tile. From there a bit of Optical Character Recognition compares each tile to a set of known examples. This works remarkably well, leading into the logic and dictionary part of the programming challenge.
Do you think this was easier or harder than the Bejeweled Blitz bot. That one was looking for specific pixel regions, this one is basically a focused roll-your-own OCR script.
I would thinking it would be easier just to match the image of a letter than using true OCR. A single letter should look exactly the same each time in this game.
I don’t remember is this particular games does it… but many of King.com’s (I assume that’s where he is playing this game) games will blur and distort individual pixels to make an exact match not work. Years and years ago I had bots running for over a dozen of their more popular games, using screen captures and simulated mouse clicks. Ahh, those were fun times.
There’s glow, particle effects, shaking tiles, text floating on top, etc.
I don’t think it’s “true” OCR in the way you’re referring to, in which it can recognize features, different fonts, and widely varying sizes.
It’s just a fuzzy match to one of the pre-stored bitmaps representing various letters. I’ve done a few game bots using this technique. On the more practical side, I once had to OCR a few hundred pages of output from a dot-matrix printer with a failing ribbon. Commercial OCR software absolutely sucked at this, but my old bot code performed wonderfully with a little tweaking.
It’s not “true” OCR, like you said; I took all the shorcuts I could, and it’s *still* failing at times. Like I mention in the article, I salute people who manage to make “real” OCR to work as well as it does. Lots of glyphs (even if you know the exact font) differ only by a couple of pixels.
I thought the sequel to Bejeweled Bot would be for Candy Crush…
I wrote a flappy bird bot, but it just centered the bird :<
“I experimented with detecting keyboard presses to kill the bot, but with the focus on the game this is rather tricky.”
Piece of cake, use the old RegisterHotKey / UnregisterHotKey.
There’s even a code sample on the msdn page :
http://msdn.microsoft.com/en-us/library/windows/desktop/ms646309%28v=vs.85%29.aspx
I knew there was an easy solution, just didn’t want to spend time looking for it =) Thanks for the hint nevertheless, may be useful in the future..