AI Beats Poker Pros: Skynet Looms

There have been a few “firsts” in AI-versus-human gaming lately, and the computers are now beating us at trivia, chess and Go. But in some sense, none of these are really interesting; they’re all games of fact. Poker is different. Aside from computing the odds of holding the winning hand, where a computer would obviously have an advantage, the key to winning in poker is bluffing, and figuring out when your opponent is bluffing. Until recently, this has helped man beat the machine. Those days are over.

Chess and Go are what a game theorist would call games of perfect information: everyone knows everything about the state of the game just from looking at the board, and this means that there is, in principle, a best strategy (series of moves) for every possible position. Granted, it’s hard to figure these out because it’s a big brute-force problem, but it’s still a brute-force problem where computers have an innate advantage. Chess and Go are games where the machines should be winning. 

In poker, you don’t know the state of the game because you can’t see your opponent’s cards. And when your opponent signals that it has good cards by betting big, your job is to interpret that signal and raise or fold. A good human poker player can get by on defensive play like this, but the really great poker players also know when to bluff. (As well as when to walk away, and when to run.) This element of strategic deception has been enough in the past to give humans the advantage over the computer’s rock-solid statistics. We’ve out-bluffed the machines.

The Carnegie Mellon team, in addition to throwing tons of computing power at the problem, focused on improving defense. Every night, the computer asked itself what its three biggest mistakes were, and modified its strategies to avoid them the next day. By improving its defense against bluffs in the early phase of the hands, it survived to the later phases, where it had the advantage that it could accurately compute the odds. This may have forced the human players to be riskier than they should before the flop.

We’re dying to see more information and analysis, but we’ll have to wait a couple of months for the paper to be published. Now though, you should put this day on your calendar — the day that the computers called our bluffs.

40 thoughts on “AI Beats Poker Pros: Skynet Looms

  1. Poker is a social game. It can be played mechanically, but the best players are good at the social aspects of the game. Did the AI use physical “tells”, or did it use betting strategy as the only “tells”.

    I could see the AI wearing the human player down by not socializing the same as a human, and over time the human get better at strategies that would match the AI.

    1. I have seen tables of pro’s get completely flumoxed by a good but not great amateur. The Pros have a method of play that assume opponents are very good, and that makes for a degree of predictability. If the AI is not playing at the same level as the pro, I can see an element of unpredictability making it mighty difficult. Rapid leaning by the AI will add to the unpredictability – like a new opponent every time they play.

      After the AI settles down for long enough, I wonder hat happens?

      1. This program uses a huge amount of hardware. I doubt that the winnings will pay for that, or even the electricity bill. Also, this program plays heads-up poker only. With multiple players, the problem becomes much harder.

        1. Not necessary – people are not all that rational when it comes to betting games to begin with. For example everyone should know that slots are weighed heavily in favor of the machine, yet people still play, and pour more money into them than any other betting game. I doubt if better play on the part of the poker programs will have much of an impact on their popularity, and as was mentioned above, the overhead is likely to be excessive for single users to profit on the other side.

      1. speaking from significant online poker industry experience – players are already getting fleeced by product algorithms to optimize for gambler’s highs and large betting. Never play online poker without a live dealer on cam.

      2. Speaking from significant experience building software in online poker – product algorithms already fleece players by hand adjustments to optimize for inducing gambler’s high and large betting. Never play online poker without a live dealer on cam.

    1. Came here to say this, If it were any other website we would be referring to it as clickbait. Can you guys tone down the titles a bit? It feels like you are one step away from posting “You won’t believe what this robot can do!” and “Top ten hackers of 2016, Wait till you see number 8!” This is what kills websites.

  2. I have a lot of python scripts sitting around to calculate “outs” and make decisions. I could make it transmit over wireless protocols even primitive PWM for cheating. Public algo from this is a big threat..

  3. Not sure if my logic is right but here’s an idea: from now on, online poker players should have a brief chat with their fellow players before putting any money on the table. Players that can’t pass the Turing test will be ostracized.If the robots get good enough to fake the chat, well, we have larger issues to deal with :)

    Second: how hard would it be to rig a human to this in meatspace? I’m thinking of the scene in the film Casino where a card cheat with an advantageous vantage point uses some kind of morse code to signal his confederate with information on the dealer’s (supposedly) unobservable card. Not a stretch to imagine someone rigging up a system like this to the AI (ann?) referenced in the article I of course didn’t read. That said, I’m keen on keeping all ten of my fingers attached to my hands :)

  4. Hackaday journalism is pretty terrible (and very clickbaity to boot). I don’t think I will continue to read the site if they continue with this. The big thing that this article misses out on is the fact that a robot can play game theory optimal poker in a lot of situations – and its not ‘artificial intelligence’ or ‘deep learning’ or whatever the buzzword of the day is. Its just a math-based algorithm based on stack sizes, pot odds, expected value etc. In a huge amount of poker situations (cash game and tournament alike) the mathematics are such that a non-exploitable strategy can be solved for; rendering it impossible to maintain an edge. I think this is what the article means by the machine ‘playing defensively’. Ive not seen any statistics, but im sure any mildly complex (non-‘intelligent’) computer program could crush the vast majority of human players – especially if they are not allowed to use lookuptables to find these equilibrium strategies.
    The variance intrinsic to poker also means that ‘winning’ isn’t as clear as chess or go – after all, what is the definition of “winning” – is a bad bet on a bad card that happens to go well winning? Does taking a bad bet to force your opponent out of the tournament count as winning?
    Very interested to read the article when its out though. But please hackaday, stop with the headlines!

  5. A lot of reporting on this (interesting) news neglects to mention that the AI was only playing heads-up. It competed against four players total, but was one-against-one with each of them.

    Beating all four at once would be an infinitely more complex problem… Let alone a ring of 9-10 competitors.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s