statistics

22 Articles

How Random Is Random?

October 10, 2019 by 10 Comments

Many languages feature a random number generator library for help with tasks like rolling a die or flipping a coin. Why, you may ask, is this necessary when humans are perfectly capable of randomly coming up with values?

[ex-punctis] was curious about the same quandary and decided to code up an experiment to test the true randomness of human. A script guesses the user’s next input from two choices, keeping a tally in the JavaScript backend that holds on to the past five choices. If the script guesses correctly, they take $1 from the user. Otherwise, the user earns $1.05.

The data from gathered from running the script with 200 pseudo-random inputs 100,000 times resulted in a distribution of correct guess approximately normal (µ=50% and σ=3.5%). The probability of the script correctly guessing the user’s input is >57% from calculating µ+2σ. The result? Humans aren’t so good at being random after all.

It’s almost intuitive why this happens. Finger presses tend to repeat certain patterns. The script already has a database of all possible combinations of five presses, with a counter for each combination. Every time a key is pressed, the latest five presses is updated and the counter increases for whichever combination of five presses this falls under. Based on this data, the script is able to make a prediction about the user’s next press.

In a follow-up statistic analysis, [ex-punctis] notes that with more key presses, the accuracy of the script tended to increase, with the exception of 1000+ key presses. The latter was thought to be due to the use of a psuedo random number generator to achieve such high levels of engagement with the script.

Some additional tests were done to see if holding shorter or longer sequences in memory would account for more accurate predictions. While shorter sequences should theoretically work, the risk of players keeping a tally of their own presses made it more likely for the longer sequences to reduce bias.

There’s a lot of literature on behavioral models and framing effects for similar games if you’re interested in implementing your own experiments and tricking your friends into giving you some cash.

Abraham Wald’s Problem Solving Lesson Is To Seek What’s Not There

August 1, 2019 by 26 Comments

You may not know the name Abraham Wald, but he has a very valuable lesson you can apply to problem solving, engineering, and many other parts of life. Wald worked for the Statistical Research Group (SRG) during World War II. This was part of a top secret organization in the United States that applied elite mathematical talent to help the allies win the war. Near Columbia University, mathematicians and computers — the human kind — worked on problems ranging from how to keep an enemy plane under fire longer to optimal bombing patterns.

One of Wald’s ways to approach problem was to look beyond the data in front of him. He was looking for things that weren’t there, using their absence as an additional data point. It is easy to critique things that are present but incorrect. It is harder to see things that are missing. But the end results of this technique were profound and present an object lesson we can still draw from today.

Continue reading “Abraham Wald’s Problem Solving Lesson Is To Seek What’s Not There” →

Automated Dice Tester Uses Machine Vision To Ensure A Fair Game

June 20, 2019 by 7 Comments

People take their tabletop games very, very seriously. [Andrew Lauritzen], though, has gone far above and beyond in pursuit of a fair game. The game in question is Star War: X-Wing, a strategy wargame where miniature pieces are moved according to rolls of the dice. [Andrew] suspected that commercially available dice were skewing the game, and the automated machine-vision dice tester shown in the video after the break was the result.

The rig is a very clever design that maximizes the data set with as little motion as possible. The test chamber is a box with clear ends that can be flipped end-for-end by a motor; walls separate the chamber into four channels to test multiple dice on each throw, and baffles within the channels assure randomization. A webcam is positioned below the chamber to take a snapshot of each “throw”, which is then analyzed in OpenCV. This scheme has the unfortunate effect of looking at the dice from the table’s perspective, but [Andrew] dealt with that in true hacker fashion: he ignored it since it didn’t impact the statistics he was interested in.

And speaking of statistics, he generated a LOT of them. The 62-page report of results from his study is an impressive piece of work, which basically concludes that the dice aren’t fair due to manufacturing variability, and that players could use this fact to cheat. He recommends pooled sets of dice to eliminate advantages during competitive play. 

This isn’t the first automated dice roller we’ve seen around these parts. There was the tweeting dice-bot, the Dice-O-Matic, and all manner of electronic dice throwers. This one goes the extra mile to keep things fair, and we appreciate that.

Continue reading “Automated Dice Tester Uses Machine Vision To Ensure A Fair Game” →

Image from official site catan.com

How Provably Loaded Dice Lead To Unprovable Cheating

December 18, 2017 by 16 Comments

Here’s a really interesting writeup by [Mike] that has two parts. He shows that not only is it possible to load wooden dice by placing them in a dish of water, but that when using these dice to get an unfair advantage in Settlers of Catan, observation of dice rolls within the game is insufficient to prove that the cheating is taking place.

[Mike] first proves that his pair of loaded dice do indeed result in a higher chance of totals above seven being rolled. He then shows how this knowledge can be exploited by a Settlers of Catan player to gain an average 5-15 additional resource cards in a typical game by taking actions that target the skewed distribution of the loaded dice.

The second part highlights shortcomings and common misunderstandings in current statistical analysis. While it’s possible to prove that the loaded dice do have a skewed distribution by rolling them an arbitrary number of times, as [Mike] and his wife do, it is not possible to detect this cheating in a game. How’s that? There are simply not enough die rolls in a game of Settlers to provide enough significant data to prove that dice distribution is skewed.

Our staff of statistics Ph.D.s would claim that [Mike] overstates his claims about shorcomings in the classical hypothesis testing framework, but the point remains that it’s possible to pass through any given statistical testing process by making the effect just small enough. And we still think it’s neat that he can cheat at Settlers by soaking wooden dice in water overnight.

This isn’t the first time we’ve seen Settlers of Catan at the center of some creative work. There’s this deluxe, hand-crafted reboot, and don’t forget the electroshock-enabled version.

[via Reddit; images from official Catan site]

Statistics And Hacking: A Stout Little Distribution

December 13, 2017 by 18 Comments

Previously, we discussed how to apply the most basic hypothesis test: the z-test. It requires a relatively large sample size, and might be appreciated less by hackers searching for truth on a tight budget of time and money.

As an alternative, we briefly mentioned the t-test. The basic procedure still applies: form hypotheses, sample data, check your assumptions, and perform the test. This time though, we’ll run the test with real data from IoT sensors, and programmatically rather than by hand.

The most important difference between the z-test and the t-test is that the t-test uses a different probability distribution. It is called the ‘t-distribution’, and is similar in principle to the normal distribution used by the z-test, but was developed by studying the properties of small sample sizes. The precise shape of the distribution depends on your sample size. Continue reading “Statistics And Hacking: A Stout Little Distribution” →

Statistics And Hacking: An Introduction To Hypothesis Testing

December 1, 2017 by 20 Comments

In the early 20th century, Guinness breweries in Dublin had a policy of hiring the best graduates from Oxford and Cambridge to improve their industrial processes. At the time, it was considered a trade secret that they were using statistical methods to improve their process and product.

One problem they were having was that the z-test (a commonly used test at the time) required large sample sizes, and sufficient data was often unavailable. By studying the properties of small sample sizes, William Sealy Gosset developed a statistical test that required fewer samples to produce a reasonable result. As the story goes though, chemists at Guinness were forbidden from publishing their findings.

So he did what many of us would do: realizing the finding was important to disseminate, he adopted a pseudonym (‘Student’) and published it. Even though we now know who developed the test, it’s still called “Student’s t-test” and it remains widely used across scientific disciplines.

It’s a cute little story of math, anonymity, and beer… but what can we do with it? As it turns out, it’s something we could probably all be using more often, given the number of Internet-connected sensors we’ve been playing with. Today our goal is to cover hypothesis testing and the basic z-test, as these are fundamental to understanding how the t-test works. We’ll return to the t-test soon — with real data. Continue reading “Statistics And Hacking: An Introduction To Hypothesis Testing” →

AI Beats Poker Pros: Skynet Looms

February 7, 2017 by 41 Comments

There have been a few “firsts” in AI-versus-human gaming lately, and the computers are now beating us at trivia, chess and Go. But in some sense, none of these are really interesting; they’re all games of fact. Poker is different. Aside from computing the odds of holding the winning hand, where a computer would obviously have an advantage, the key to winning in poker is bluffing, and figuring out when your opponent is bluffing. Until recently, this has helped man beat the machine. Those days are over.

Chess and Go are what a game theorist would call games of perfect information: everyone knows everything about the state of the game just from looking at the board, and this means that there is, in principle, a best strategy (series of moves) for every possible position. Granted, it’s hard to figure these out because it’s a big brute-force problem, but it’s still a brute-force problem where computers have an innate advantage. Chess and Go are games where the machines should be winning.  Continue reading “AI Beats Poker Pros: Skynet Looms” →