Finally, a useful application for machine vision! Forget all that self-driving nonsense and facial recognition stuff – we’ve finally got an AI that can count cards at the blackjack table.
The system that [Edje Electronics] has built, dubbed “Rain Man 2.0” in homage to the classic title character created by [Dustin Hoffman] for the 1988 film, aims to tilt the odds at the blackjack table away from the house by counting cards. He explains one such strategy, a hi-low count, in the video below, which Rain Man 2.0 implements with the help of a webcam and YOLO for real-time object detection. Cards are detected in any orientation based on their suit and rank thanks to an extensive training set of card images, which [Edje] generated synthetically via some trickery with OpenCV. A script automated the process and yielded a rich training set of 50,000 images for YOLO. A Python program implements the trained model into a real-time card counting application.
Rain Man 2.0 is an improvement over [Edje]’s earlier Tensor Flow card counter, but it still has limitations. It can’t count into a six-deck shoe as the fictional [Rain Man] could, at least not yet. And even though cheater’s justice probably isn’t all cattle prods and hammers these days, the hardware needed for this hack is not likely to slip past casino security. So [Edje] has wisely limited its use to practicing his card counting skills. Eventually, he wants to turn Rain Man into a complete AI blackjack player, and explore its potential for other games and to help the visually impaired.
Is the title misspelled?
Missed the man point
D’oh!
So now we’re going to have robots buried out in the desert?
That’s amazing. And all those tricks to generate the training images. Very cool project.
Counting cards doesn’t change the odds, it tells you when to bet big.
If counting cards makes it less likely that the house walks away with your money, then the odds are ever in your favor…
I’ve been waiting for casinos to start implementing something like this in their card shuffling machines, especially those continuous shuffle ones where the full shoe never leaves the machine.
What happens if gaming commission inspectors find cameras in the shufflers? “Oh those are to make sure there aren’t any card jams.”
There is no transparency with shuffling machines. You can’t watch the cards being shuffled like you can watch a dealer to make sure that nothing funny is going on. Might as well let the dealer set up a screen to hide behind while they shuffle.
Is this paranoid?
Would it be worth it to casinos to try this and risk being caught? ABSOLUTELY.
Just because you’re paranoid doesn’t mean they aren’t following you.
In the end, this wouldn’t affect me; I don’t gamble. My average number of hands at a table is equal to (my buy in/min bet) + 1, even if I’m following the proper betting process. I’m just that unlucky.
Err, hate it when it does this. That was supposed to be a new comment.
I’ve never seen the inside of a shuffling machine, but it seems like it would be difficult to make room for a camera and lighting while still providing enough space for the camera to “see” each card. Would there have to be a mechanism that passes each and every card in front of the camera and then decides where to put it in the shuffle? How would that work?
Also, they offer players the option to cut the deck wherever they want: the player can move all the bottom cards to the top of the shoe, or vice versa. That would undo any attempt at putting all the ten-cards at the bottom of the shoe.
Regarding your playing experience… dang, that IS unlucky! Maybe they really are stacking the deck :)
Vision systems are tiny now a days, and some shuffling machines are installed into (under) the table itself. They can be totally unseen except for a ‘drawer’ where the cards go that pops up above the table when a button is pushed.
Continuos shuffling machines do not allow for a player to cut the shoe. The dealer shows that the decks are complete before putting all of the cards into the machine (this only happens once at the beginning). The machine shuffles, then a hand is dealt. At the end of a hand the cards are put into the machine again. Repeat until the house rules require or the pit boss signals for new deck(s).
To answer your sorting questions:
Yes, the machine would have to be able to move cards in different directions. For example, sort them into two piles: one to be shuffled and dealt normally and the other to be removed from play.
As far as how they’d affect the game, simply removing face cards or 10s from play will make the house less likely to bust. It doesn’t take much to make an impact.
How to do this, you ask? I can think of at least one easy way to do this which would result in cards being put into two piles using only one DC motor (run forward or backward depending on what pile the software wants the card to be in). This motor would have to be in line with the rest of the shuffling mechanism, too. I don’t have time to explain it right now, but if I’m feeling ambitious later I may try to follow up on this.
I’ve been playing with this in my head periodically for the past 4 or 5 years (my in-laws play black jack compulsively on vacations). If I wasn’t so horrible at following through on things, I would try to make an automated ‘card mechanic’ shuffler, but realistically I’m never going to follow through on that…
Nice work-
Thanks for sharing the process.
Oh hey! I’m a bit late, but this is a pretty specific area where I happen to have a lot of in-depth knowledge from working in the industry for three decades. I’ve got a few notes here:
1. There is only one real shuffle machine manufacturer whose name is very easy to find, but I won’t mention here. There are a couple of smaller shuffler vendors, and several different companies manufacturer intelligent shoes for baccarat, but for shufflers the one dominant company has the vast majority of market share in the western world at least.
2. Shufflers and intelligent shoes have had cameras inside them for many years now. Shufflers read the cards on insertion and shoes read them on removal. For shufflers this is used for ensuring the decks have the correct cards. A feature you’ll find interesting is that some of these shufflers include a feature that will sort the deck back into the order it comes out of the pack. This can be incredibly useful when closing a game or changing cards. The intelligent shoes read the cards as they are removed to inform an attached display of the Baccarat hand outcome and optionally the shoe may stop delivering cards when the hand is complete to prevent accidental dealer overdraws which can be a big pain in the butt.
3. You’re correct that there’s no transparency when it comes to shufflers. That does suck. However, the patents are all out there that explain in pretty good detail exactly how they work. Much like in crypto, good shuffling solutions don’t require anything to be hidden, technique-wise. I know that to a skeptical/paranoid person that doesn’t offer much relief; but for what little it’s worth, I the random-internet-guy can tell you that they do work as depicted in the patents.
4. Gaming Commissions in every jurisdiction in America require the software for shufflers (and slots, and any other electronic device that is part of a gambling game) to be verified by one of two independent testing agencies, GLI or BMM. Every single revision requires extremely expensive and thorough testing. Software is then sent with a GLI/BMM approval letter listing the SHA of the software to the Gaming agents who verify the SHA is correct before the casino is allowed to install that software. Many jurisdictions also randomly audit the SHAs of installed software on the floor during unannounced visits. I can’t speak for the tribal casinos, but Nevada and Jersey are pretty strict about that sort of stuff. The tribes are governed by their respective states’ gaming commissions on top of their own internal gaming commissions both of which generally model after Nevada’s policies.
5. Since it’s within the capabilities of the machine to order cards as it sees fit, let’s talk about the practical applications of actually doing this:
A: The shuffle machine manufacturer needs to write the custom software to allow for cheating. I suppose a casino with the resources and wherewithal could hire an expensive tech to reverse engineer the machine and write custom software, but damn is that incredibly unlikely. The manufacturer just plain would never do it. The shuffle machines are a very small part of their $2 BILLION market cap. If they were ever caught doing that they’d never be able to sell a gaming product in the US again. It’s just not worth it to them.
B: The casino needs to have a surprisingly large number of people involved in this conspiracy to cheat. You need a tech to receive the software and install it, which isn’t HARD, but it’s non-trivial. Your director of Table Games is obviously going to be in on it, and possibly upper management. He’ll need to inform, at a minimum, his three shift managers AND their #2s, so that if they see a gaming agent on the floor they can quickly go swap those shufflers out with the right software. Your dealers, the highest turnover job in the industry, and who work for tips that generally come from winning players, will probably need to know too for the next reason:
C: In the case of blackjack, the shuffler will need to know at least one hand in advance how many players will be playing the next hand so it can steer the cards correctly. This is a big stretch, but many casinos do have RFID chips inside their gaming chips and sensors under the spots for average bet calculation and/or hands-per-hour metrics, so I guess it’s not outside the realm of possibility. Worth noting that that large shuffler manufacturer does not offer that sort of RFID product. The reason it needs to know a hand in advance is that all of the continuous shufflers deliver plugs/packs of between 8-17 cards at a time to lip of the shoe so it will need to have those basically pre-prepared. Don’t have to take my word on that; patents are all out there.
D: If a player ever caught on this, the best outcome is that they go to the authorities. Another possible outcome is that they realize how easy it is to take advantage of a shuffler that delivers card deterministically and take you for tons of money. You also need to worry about anyone in your casino, or worse, someone outside the casino, who knows how this scam works and takes advantage via employee/guest collusion.
Since I’m writing a novel here I will mention that on a shoe game with a cut card there is a way to get around needing to know the number of players at the table and how will they play. You can pre-arrange the deck so that a large number of tens and aces fall behind the cut card, which can substantially increase the house edge. However, you’re still facing all of the other problems, and depending on the game rules might still be vulnerable to someone taking advantage if they know about it.
I hope all of that puts your mind a bit at ease about casinos using technological means to cheat. That said, if the casino really did want to cheat its players, there’s a way, way way, way, way, way easier method that’s been known for decades. Each ten-valued card is worth about (+0.5% / NumOfDecks) expectation to the player. This is the cornerstone for card counting. Now, let’s say you had a double deck game and you removed four ten valued cards at random. You’ve just increased the house edge by (0.5% / 2) * 4 = 1%. On a six deck game with decent rules the player starts at about a 0.5% disadvantage depending on their ability. You could easily triple that by simply removing 12 ten valued cards. This can be done discretely with very few people in on it, and little risk of discovery. This is one of many reasons I would never play in a small, foreign casino.
But in reality, no blackjack table, at least in the US, is ever going to rigged in the modern corporate-controlled gambling industry. Tables only represent about 10% of the overall revenue of the casino so it’s just plain not worth it. Especially since the largest revenue generating area, slots, allows you to arbitrarily set payback percentages to whatever you’d like with no notification to the guest (with the exception of video poker and keno which must play to the proper odds, but they’re small potatoes).
By any chance is what you have for sale? That would be great for a magic routine for magicians. I do a complete rain man routine and perform that same effect in my head without electronics. But I would be interested in playing with this if you make it available