There have been a few “firsts” in AI-versus-human gaming lately, and the computers are now beating us at trivia, chess and Go. But in some sense, none of these are really interesting; they’re all games of fact. Poker is different. Aside from computing the odds of holding the winning hand, where a computer would obviously have an advantage, the key to winning in poker is bluffing, and figuring out when your opponent is bluffing. Until recently, this has helped man beat the machine. Those days are over.
Chess and Go are what a game theorist would call games of perfect information: everyone knows everything about the state of the game just from looking at the board, and this means that there is, in principle, a best strategy (series of moves) for every possible position. Granted, it’s hard to figure these out because it’s a big brute-force problem, but it’s still a brute-force problem where computers have an innate advantage. Chess and Go are games where the machines should be winning.
In poker, you don’t know the state of the game because you can’t see your opponent’s cards. And when your opponent signals that it has good cards by betting big, your job is to interpret that signal and raise or fold. A good human poker player can get by on defensive play like this, but the really great poker players also know when to bluff. (As well as when to walk away, and when to run.) This element of strategic deception has been enough in the past to give humans the advantage over the computer’s rock-solid statistics. We’ve out-bluffed the machines.
The Carnegie Mellon team, in addition to throwing tons of computing power at the problem, focused on improving defense. Every night, the computer asked itself what its three biggest mistakes were, and modified its strategies to avoid them the next day. By improving its defense against bluffs in the early phase of the hands, it survived to the later phases, where it had the advantage that it could accurately compute the odds. This may have forced the human players to be riskier than they should before the flop.
We’re dying to see more information and analysis, but we’ll have to wait a couple of months for the paper to be published. Now though, you should put this day on your calendar — the day that the computers called our bluffs.
Poker is a social game. It can be played mechanically, but the best players are good at the social aspects of the game. Did the AI use physical “tells”, or did it use betting strategy as the only “tells”.
I could see the AI wearing the human player down by not socializing the same as a human, and over time the human get better at strategies that would match the AI.
Just betting strategy.
The other thing is that it seems the computer only called bluffs, never issuing them.
I believe, from the CMU website, that it also bluffed. They just spent most of their effort on calling them.
I have no idea how you would write an optimal bluffing algorithm. As I wrote, I can’t wait to read the paper.
I assume by a form of card counting you’d know when the deck had gone cold and opponents unlikely to have really good hands, hence more vulnerable to bluff.
Not sure how much advantage a computer has counting up to seven….maybe if it had a thermocouple or something.
@Jerry A thermal camera would be its secret weapon.
I have seen tables of pro’s get completely flumoxed by a good but not great amateur. The Pros have a method of play that assume opponents are very good, and that makes for a degree of predictability. If the AI is not playing at the same level as the pro, I can see an element of unpredictability making it mighty difficult. Rapid leaning by the AI will add to the unpredictability – like a new opponent every time they play.
After the AI settles down for long enough, I wonder hat happens?
A year from now, bots like this will be earning pretty much all the money from online pokers sites, and two years from now, those poker sites will have died because they lost all interest to players.
Thanks, robots !
Thinking about it, I’d bet some students working on this are getting very rich right now …
This program uses a huge amount of hardware. I doubt that the winnings will pay for that, or even the electricity bill. Also, this program plays heads-up poker only. With multiple players, the problem becomes much harder.
I am not sure you fully understand how much money is involved in online poker.
If you only play a single game at a time, you can’t win more than your opponent is willing to lose. How much is that ?
Well, looking at the data, officially, somewhere around 450 billion annually for all gambling worldwide.
That’s great, but the total is irrelevant if only playing a single game at a time already costs a huge amount of hardware.
@Arthur Wolf – What makes you think those sites won’t be using these bots to fleece the human players?
The point is, if users can use those bots, doesn’t matter if the site themselves do or not, the users that don’t use bots essentially see their chances of winning go way down, and the site becomes way less attractive.
Not necessary – people are not all that rational when it comes to betting games to begin with. For example everyone should know that slots are weighed heavily in favor of the machine, yet people still play, and pour more money into them than any other betting game. I doubt if better play on the part of the poker programs will have much of an impact on their popularity, and as was mentioned above, the overhead is likely to be excessive for single users to profit on the other side.
speaking from significant online poker industry experience – players are already getting fleeced by product algorithms to optimize for gambler’s highs and large betting. Never play online poker without a live dealer on cam.
Only if your name is Peter Ness.
https://www.youtube.com/watch?v=55XtHV_vwcY
Haha! Lets not forget ‘mike hawk’ and ‘mike hunt’……though those might be a bit more offensive!
Speaking from significant experience building software in online poker – product algorithms already fleece players by hand adjustments to optimize for inducing gambler’s high and large betting. Never play online poker without a live dealer on cam.
I wouldn’t gamble online any more than I gamble off, and for the same reason – the odds are not in my favor.
just YES.
As if one could trust an online poker site…
Online poker is already over-run with bots though, isn’t it?
Skynet this, Skynet that…
We get it, the movie, Terminator, is jesus lube. How many times are you going to nuke the planet with skynet references?
Okay, Matrix references instead.
One of these days I will write a browser plugin that will eliminate all Skynet references in the text,
FoxReplace does this.It only changes your web page, but that’s a start.
comma horror
Came here to say this, If it were any other website we would be referring to it as clickbait. Can you guys tone down the titles a bit? It feels like you are one step away from posting “You won’t believe what this robot can do!” and “Top ten hackers of 2016, Wait till you see number 8!” This is what kills websites.
When you lack any other reference because you´ve been fed Hollywood movies your whole life, that´s what happens. Next step: Idiocracy.
This is heads up only. One vs one. It’s a bit more difficult with many opponents.
I have a lot of python scripts sitting around to calculate “outs” and make decisions. I could make it transmit over wireless protocols even primitive PWM for cheating. Public algo from this is a big threat..
Not sure if my logic is right but here’s an idea: from now on, online poker players should have a brief chat with their fellow players before putting any money on the table. Players that can’t pass the Turing test will be ostracized.If the robots get good enough to fake the chat, well, we have larger issues to deal with :)
Second: how hard would it be to rig a human to this in meatspace? I’m thinking of the scene in the film Casino where a card cheat with an advantageous vantage point uses some kind of morse code to signal his confederate with information on the dealer’s (supposedly) unobservable card. Not a stretch to imagine someone rigging up a system like this to the AI (ann?) referenced in the article I of course didn’t read. That said, I’m keen on keeping all ten of my fingers attached to my hands :)
Hackaday journalism is pretty terrible (and very clickbaity to boot). I don’t think I will continue to read the site if they continue with this. The big thing that this article misses out on is the fact that a robot can play game theory optimal poker in a lot of situations – and its not ‘artificial intelligence’ or ‘deep learning’ or whatever the buzzword of the day is. Its just a math-based algorithm based on stack sizes, pot odds, expected value etc. In a huge amount of poker situations (cash game and tournament alike) the mathematics are such that a non-exploitable strategy can be solved for; rendering it impossible to maintain an edge. I think this is what the article means by the machine ‘playing defensively’. Ive not seen any statistics, but im sure any mildly complex (non-‘intelligent’) computer program could crush the vast majority of human players – especially if they are not allowed to use lookuptables to find these equilibrium strategies.
The variance intrinsic to poker also means that ‘winning’ isn’t as clear as chess or go – after all, what is the definition of “winning” – is a bad bet on a bad card that happens to go well winning? Does taking a bad bet to force your opponent out of the tournament count as winning?
Very interested to read the article when its out though. But please hackaday, stop with the headlines!
What isn’t “clickbaity” in 2017? Everyone uses attractive images and headlines for almost opposite content.. They do it for revenue..
A week later, and still no one has mentioned that the best poker players also know not to count their money while still at the table?
A lot of reporting on this (interesting) news neglects to mention that the AI was only playing heads-up. It competed against four players total, but was one-against-one with each of them.
Beating all four at once would be an infinitely more complex problem… Let alone a ring of 9-10 competitors.
The same way you’d beat a good human player: play better poker than them. There are no GTO NLHE/PLO bots as of yet.