Automating Mobile Games With A Robot Arm

My Singing Monsters is one of those mobile titles that has users play simple games to earn coins and gems in the usual way. [Anykey] found that his son was a fan of the game, but that sometimes it felt a little rigged. Thus, rather than waste time playing themselves, he set up a robot to do the job for them. (Super-boring video, embedded below.)

The player must complete a basic but time-consuming memory game. Upon winning, the player gets to choose a prize from 17 mystery cards. The top prize of 1,000 diamonds always seemed to be hidden under another card, leading to the aforementioned frustration.

In order to test if the game was rigged, [Anykey] set up a uArm Swift Pro to play the game, with the robot arm moving a small stylus over the iPad playing the game. The iPad’s video was piped to a PC via HDMI out, going into a Camlink capture card. A Python script using OpenCV was then created to play the game automatically, and log the results of prizes gained along the way. All the code is up on GitHub.

After over 100 attempts, the robot never managed to pick the right card to score 1,000 diamonds. Given that there are only 17 cards to choose from, one would expect the 1,000 diamond prize to come up several times in that many selections.

It seems then that the prize selection for completing the memory game may not actually be down to picking the right card. Instead, the prize given is selected by some other calculation entirely.

We love a robot playing games at Hackaday, even if it’s as simple as Tic-Tac-Toe. Video after the break.


16 thoughts on “Automating Mobile Games With A Robot Arm

    1. That is a correct assessment :)

      As you might expect, the big ticket winning item (1,000 Diamonds) is not assigned to a card prior to letting the player pick a card; If it did, you would have a 1-in-17 chance of getting it.

      Instead, the game shows 17 cards, *implying* (but never explicitly stating) that you could be lucky and pick the winning card.. when in reality the game has already picked what you’re actually going to win, and whatever card you pick.. it will show those winnings while distributing the remainder of the possible prizes randomly across the remaining cards.

      That’s not the only way in which it is rigged, either. Here’s a deep dive video if this sort of thing interests you at all:

      One slight note on the use the word ‘rigged’ as I’ve received some feedback from players saying “It’s not rigged, I got 100 Diamonds from it!”. Rigged within the context of this comment (and the video linked) is with regard to the odds not lining up with expectations based on the presentation, and with regard to the nature of the minigame.

    2. That assessment is correct :)

      The game essentially picks the prize for you before you even tap on one of the cards at the ‘end’ of the minigame. Whatever you tap will be that prize, and remaining potential prizes are distributed randomly among the cards you didn’t tap. I made a video that goes into a bit more detail if anyone’s interested; “aimeeplaysmsm memory rigged” should pop it right up.

      It IS possible to get the top reward (1,000 Diamonds, a valuable in-game currency), just incredibly unlikely, and certainly not a 1-in-17 chance :D

      ( if my response using twitter auth ever comes through: apologies for multiple replies )

    3. It depends on what you mean by “rigged”.

      Almost always, the chance of winning something is configured in the game and has nothing to do with what you see on screen. (Just like in a casino.)

      So, with 17 cards, the chance of winning 1,000 diamonds (or anything else) is not necessarily 1 in 17, although the game might give you that impression.

      The chance of winning diamonds might be 1 in 100,000. Winning something else might be 1 in 2, etc…
      Choosing any card from the stack of 17 just starts this random pick and it doesn’t actually matter which card you picked in the first place.

      If the game uses these configured non-zero odds, even incredibly small, it is not rigged.
      If the game never gives you diamonds, it is rigged.

      Off course, as is common in such games, the odds of hitting the jackpot are not really in your favor, but the gameplay is designed to keep you trying.

      Here in the EU, in certain situations such random prizes in games are considered to be a form of lottery/gambling. (Because it is.)
      As this kind of gameplay is actually pretty addictive for young players (and some adult people too) it is kind of regulated in what can and can’t be done in the game, especially when real money is involved.

      1. Wouldn’t that make it impossible to prove any game to be rigged? If the operator promises there’s a chance of winning but conceals the odds, how can anyone prove the odds are non-zero?

        1. How do you “prove” the bullet in your 1st person shooter game really did/didn’t hit the target?

          If it’s a regulated industry ==> Regulation and audits.
          If it isn’t a regulated industry ==> Trust in the vendor / Open Source code.

          The odds can always be discovered by statistical analysis (but hard/impossible to do when playing the game requires real money).
          Another way is reverse engineering / analysing the software.

          Most trustworthy operators do publish the odds for such games, although that info might be buried deep on their website.

          FYI: If random items are received after a payment, the odds must even be disclosed (by most app stores). Ex: Apple states:

          Apps offering “loot boxes” or other mechanisms that provide randomized virtual items for purchase must disclose the odds of receiving each type of item to customers prior to purchase.


          1. Just look at how the mobile games industry is doing. As an example, the games from Vivendi / Gameloft. Which is also a media oligopoly. How would you feel if an online casino is controlling the news you see?

            This illegal gambling industry is in dire need of regulation, once they accept payments of any sort. It’s currently a wild west, just like the crypto “industry”.

            I’m all for decentralization and deregularization, but this is not the case.You have corporations preying on gambling instincts. This is beyond safety warning such as ‘do not eat’. The washing pods do not have a writing on them such as ‘tasty, forest fruit flavor’ and in small print ‘might cause discomfort when eaten’.

  1. At first I thought, “if you can’t be bothered to play the game, don’t play.” Then I read the part about wanting to figure out if it was rigged, and then I thought, “ahh, good idea.”

    Clearly it’s rigged. I have a feeling every computerized gambler is rigged. In vegas they used to have these older video black jack machines that seemed to follow VERY clear and easy to discern patterns. They’d let you win a few small hands, then they’d give you a big win, then immediately after the big win they’d take everything you bet. I didn’t have a huge test set, cause money, but it wasn’t something that happened sometimes. You could easily wait for that first big payout, then switch machines and get it again. If you didn’t switch, they took whatever your next bet was (dealer black jacks and you either get lower or bust). After a big win, it’s usually a bigger bet than your previous bets. I hate gambling, and didn’t know the exact size of what constitutes a “big bet” so I didn’t bother trying to game the system. Plus they kick you out for winning too much, and I was with friends. I think I won about $30 then quit trying. If you can still find them, they’re the machines with the old square red buttons for which card you want to keep or draw and the CRT screens with graphics that look at home on a DOS game.

  2. “The Electric Monk was a labour-saving device, like a dishwasher or a video recorder. Dishwashers washed tedious dishes for you, thus saving you the bother of washing them yourself, video recorders watched tedious television for you, thus saving you the bother of looking at it yourself; Electric Monks believed things for you, thus saving you what was becoming an increasingly onerous task, that of believing all the things the world expected you to believe.”

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.