[Gadget Addict] found out about a contest being held by a shoe seller. Their mobile app has a game very much like Bejeweled. The high scorer each month gets £500. His choices were to be better at the game than everyone else, or to be smarter. He chose the latter by writing a computer vision program to play the game.
There are two distinct parts of a hack like this one. The first is just figuring out a way to programmatically detect the game board and correctly identify each icon on it. This is an iPad game. [Gadget Addict] is mirroring the screen on his laptop, which gives him easy access to the game board and also allows for simulated swipes for automatic play. Above you can see two examples where black pixels may be counted in order to identify the icon. A set of secondary checks differentiates similar entries after the first filtering. The other part of the hack involves writing the algorithms to solve for the best move.
If you liked this one, check out a super-fast Bejeweled solver from several years back. We should also mention that this was just a proof of concept and [GA] never actually entered the contest.
Check out this brainy bot with [Jari] whipped up to dominate the Bookworm Deluxe scoreboard. The bot runs on top of a win32 machine, pulling screenshots to see the game board and simulating mouse clicks to play. The video after the jump shows that it plays like a champ, but it took some doing to get this far and [Jari] took the time to share all of the development details.
The hardest part of writing these types of bots is recognizing the game pieces. Check out all of the animation that’s going on in the still shot above… a lot of the tiles are obscured, there are different colors, and the tiles themselves shift as the bot spells and submits each word.
After some trial and error [Jari] settled on an image pre-processor which multiplies pixel values by themselves four times, then looks at each pixel with a 1/6 threshold to produce a black and white face for each tile. From there a bit of Optical Character Recognition compares each tile to a set of known examples. This works remarkably well, leading into the logic and dictionary part of the programming challenge.
Do you think this was easier or harder than the Bejeweled Blitz bot. That one was looking for specific pixel regions, this one is basically a focused roll-your-own OCR script.
Continue reading “Bookworm Playing Bot Tests Programmer’s OCR Skills”
[Michael Mitchell] put together a demonstration of how Google Talk can be used to communicate with scripts. Although the concept isn’t new we haven’t seen very many projects that use the chat interface for issuing commands and receiving data. The one that does come to mind is this home automation project which uses Google Talk because it’s quite a bit faster than SMS or email communications.
Luckily there’s already a Python library called pygtalkrobot which helps with the XMPPPY protocol used by Google Talk. In addition to that package, [Michael] also installs some extras which allow him to access the GPIO pins on the RPi via Python. In the video after the break he demonstrates switching and LED on and off, as well as reading from a slide switch connected to pin 8. Of course it’s a snap to code feedback from the OS itself. As you can see in the image above the RPi is reporting it’s uptime after being issued a command by [Michael]
Continue reading “Google Talk bot running on Raspberry Pi”
What do you put on your pancakes? Butter and syrup but not a pair of shoes? This makes sense to us, and it’s the premise of the new CAPTCHA game PlayThru. The space that is normally filled by nearly illegible text is now taken up by a little graphic-based game where you drag the appropriate items to one part of the screen. In addition to being easier than deciphering letters, this new platform shouldn’t require localization. But alas, it seems the system is already broken. [Stephen] sent us a link to a bot that can pass the PlayThru CAPTCHA.
Take a look at the video after the break to see the four test-runs. It looks like the bot is just identifying the movable objects and trying them out. Sometimes this is quick, sometimes not. But it does eventually succeed. For the PlayThru developers this should be pretty easy to fix, just make an error limit for trying the wrong item. At any rate, we can’t think defeating the current system is nearly as hard as defeating reCaptcha was.
Update: [Tyler] over at Are You A Human wrote in to share their side of this story. Apparently we’re seeing the bot play the game, but not necessarily pass it. It isn’t until the game if finished and the playing information is sent to their servers that a decision is made on whether it is successful or not. This way they can change the authentication parameters from the server side at any time.
At the same time, [Stephen] updated his bot and made a video of it playing the game without any shoes on the pancakes.
Continue reading “CAPTCHA bot beats new Are You A Human PlayThru game”
We find the programming challenge of game-playing bots to be fascinating. Take a look at this Python bot which plays Burrito Bison all the way through (video after the break). This is a totally pedantic exercise which has no purpose, other than to hone your mastery of a certain programming problem. And to that we say Bravo!
We looked in on a similar project which used some C# code to dominate the game Bejeweled Blitz. We’re not fantastic at C# and that code was never made public. But [Audionatics] has released this code through Github, and it’s written in Python which is a language in which we’re well versed.
The script monitors pixel locations to use as an input, which [Audionautics] admits is very error-prone. But if everything is setup just right it works like a charm. He’s also using the PyWin package which we believe is what lets the script move the cursor and register button clicks. We think this is really fun, but it make us wonder about the black-hat possibilities. What are the chances this could be turned into a gambling bot? Scary thought, huh?
Continue reading “Web game bot coded with Python”
Programmers don’t need to get good at a game to achieve a high score, they code a bot for that instead. Take [hypnotizd] for instance. He was learning to write in the C# language and decided to make a bot that plays Bejeweled Blitz on Facebook. He figures he took between 48 and 72 hours of coding over a couple of weeks, but remember, he was learning the language at the same time. We think you’d be hard pressed to achieve a 1.5 million range score by yourself, even with that amount of practice time.
We spoke with [hypnotizd] yesterday afternoon to get a bit of background on how he made this happen. His code (he’s not releasing it so you’ll have to write your own) scrapes the screen image as input. You can see at the beginning of the video after the break that he sizes his app to properly align each jewel in its grid. The program then identifies each game piece by finding the center of the cell and taking a 25 square-pixel average color. Many of the jewels are easily recognized in this first pass, but some are harder and require several different tests to identify. That’s the difficult part, choosing the best move is just a matter of coming up with your own rules on how the bot should play the game.
Continue reading “Bejeweled Blitz bot makes your high score look just sad”
Google’s Maker Faire exhibit space is swarmed with robots…er, androids. Amidst some cool bipeds and Segway-balancers, our inner sci-fi nerd was most smitten with this hexapod design, which they’ve dubbed SKPR Bot. The “Skipper” is on hand to showcase the ease of various Google technologies: SketchUp, Android OS and the Android Open Accessory Development Kit. The whole project came together in less than six weeks.
18 servos are mounted to a framework designed in SketchUp and laser-cut by Ponoko. The low-level servo PWM control is handled by the Dev Kit (essentially a rebadged Arduino Mega, as we’ve seen), while an Android OS phone provides a slick GUI and handles all the inverse kinematics calculations required as the robot takes each step. The coolest bit is that it’s all up for grabs. At this moment you’ll have to scrounge around the ’net a bit to find the plans and code, but some time post-Faire they plan to bring everything together at the SKPR Bot site.