Bookworm Playing Bot Tests Programmer’s OCR Skills

bookworm-bot

Check out this brainy bot with [Jari] whipped up to dominate the Bookworm Deluxe scoreboard. The bot runs on top of a win32 machine, pulling screenshots to see the game board and simulating mouse clicks to play. The video after the jump shows that it plays like a champ, but it took some doing to get this far and [Jari] took the time to share all of the development details.

The hardest part of writing these types of bots is recognizing the game pieces. Check out all of the animation that’s going on in the still shot above… a lot of the tiles are obscured, there are different colors, and the tiles themselves shift as the bot spells and submits each word.

After some trial and error [Jari] settled on an image pre-processor which multiplies pixel values by themselves four times, then looks at each pixel with a 1/6 threshold to produce a black and white face for each tile. From there a bit of Optical Character Recognition compares each tile to a set of known examples. This works remarkably well, leading into the logic and dictionary part of the programming challenge.

Do you think this was easier or harder than the Bejeweled Blitz bot. That one was looking for specific pixel regions, this one is basically a focused roll-your-own OCR script.

 

Comments

  1. Shanee says:

    I would thinking it would be easier just to match the image of a letter than using true OCR. A single letter should look exactly the same each time in this game.

    • Jay says:

      I don’t remember is this particular games does it… but many of King.com’s (I assume that’s where he is playing this game) games will blur and distort individual pixels to make an exact match not work. Years and years ago I had bots running for over a dozen of their more popular games, using screen captures and simulated mouse clicks. Ahh, those were fun times.

    • Chris C. says:

      I don’t think it’s “true” OCR in the way you’re referring to, in which it can recognize features, different fonts, and widely varying sizes.

      It’s just a fuzzy match to one of the pre-stored bitmaps representing various letters. I’ve done a few game bots using this technique. On the more practical side, I once had to OCR a few hundred pages of output from a dot-matrix printer with a failing ribbon. Commercial OCR software absolutely sucked at this, but my old bot code performed wonderfully with a little tweaking.

      • It’s not “true” OCR, like you said; I took all the shorcuts I could, and it’s *still* failing at times. Like I mention in the article, I salute people who manage to make “real” OCR to work as well as it does. Lots of glyphs (even if you know the exact font) differ only by a couple of pixels.

  2. Alan says:

    I thought the sequel to Bejeweled Bot would be for Candy Crush…

  3. syntroniks says:

    I wrote a flappy bird bot, but it just centered the bird :<

  4. pac says:

    “I experimented with detecting keyboard presses to kill the bot, but with the focus on the game this is rather tricky.”
    Piece of cake, use the old RegisterHotKey / UnregisterHotKey.
    There’s even a code sample on the msdn page :
    http://msdn.microsoft.com/en-us/library/windows/desktop/ms646309%28v=vs.85%29.aspx

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 92,285 other followers