The "bite size" format of articles is meant to deliver concise, focused insights on a single, small-scope topic. My goal is to write an article that gives you a few key takeaways that you could read during a quick break at work. You’ll understand these key points after reading this article:
- The definition of the gambler’s fallacy
- Why we fall for it
- The problems it can cause in you work as a data scientist and how to avoid those problems
1 – What is the gambler’s fallacy?
The gambler’s fallacy is the incorrect assumption that prior random events will impact other random events. It is a Cognitive Bias that causes us to believe that what randomly happened before will influence future random outcomes. The opposite of this fallacy is understanding that randomness is random and no number of peculiar independent events before a random event influence the event itself.
Let’s get some intuition on this fallacy with a few examples:
Coin Toss (the "classic" example)
The classic and thoroughly used example for the gambler’s fallacy is a series of coin tosses. Suppose that I flip a coin nine times – all of the nine flips have been ‘heads.’ I’m about to flip it for the tenth time. What is the probability of getting ‘heads’ again? The correct answer (assuming the flips are independent and the coin is ‘fair’) is 50%. This can often feel wrong because it seems so unlikely to flip 10 heads in a row. That ‘wrong’ feeling is the gambler’s fallacy working inside of you!
Roulette Table
Imagine you are gambling at a casino – you are at the roulette table and you’ve been on black for the last 10 spins (black means you get a payout if the spin lands on any black number; which has about a 48% probability), you have lost every spin. Should you stay on black? Or go to red? Or leave the casino with your head down? If you are like me and many people, something inside of you wants to stay on black because it is ‘due’ for a win. It just feels so unlikely that the next spin would not be black again. That feeling is the gambler’s fallacy!
Traffic Lights
Imagine you are driving to work and you hit three red lights in a row – let’s assume that these traffic lights operate independently of each other (probably not a fair assumption for real life). Frustrated, you know that the next light has to be green, how could you get stuck in so many red lights in the same commute!? This is also the gambler’s fallacy.
Sport Performance
Imagine now that you are watching your favorite sports team (mine is the Dallas Mavericks). Your star player has made 10 shots in a row, he is on fire! Your friend, a fan of the opposing team, looks over to you and says – he’s about to start missing a bunch of shots because his shooting average is 47%. Assuming that your star player’s shots are independent of each other, your friend’s conclusion is unwarrented. Your friend has been a victim of the gambler’s fallacy!
2 – Why do we fall for the gambler’s fallacy?
Now that we understand what the gambler’s fallacy is, let’s talk about why it appeals to us. There are many factors and theories about what makes us subject to the gambler’s fallacy. I wanted to focus on two of the reasons I find most interesting and compelling.
We misunderstand the law of large numbers
The law of large numbers asserts that as a sample size gets large, the sample mean gets very close to the population mean (this is an informal defintion). The concept that larger samples increases the probability that the sample mean matches the population mean is intuitive and correctly understood by most people. The misunderstanding is how the sample mean converges to the population mean.
Correct understanding: As sample size increases, the volume of observations ‘dilute’ the influence of extreme random values on either side. This dilution balances out both sides of the sample distribution and will eventually result in the convergence of the sample mean with the population mean.
Incorrect understanding: We often accidentally think (usually implicitly) that the law of large numbers is ‘enforced’ by some kind of a equilibrium mechanism. E.g., if we had an extremely low observation in our sample, we will have an extremely high observation in our sample to compensate. Of course, there is no ‘equilibrium’ mechanism – this is a misunderstanding of the nature of random events. Sample observations don’t offset each other, they dilute each other. If we have a very large sample size, it is probable that we would have extremely low observations to balance out the extremely high observations – but there is no rule that enforces it. That is why the sample size matters so much in the law of large numbers. The more random samples we take, the more likely observations are to balance each other out. If there was an equilibrium mechanism, sample size wouldn’t be as important because balanced samples would be enforced. This implicit belief that there is a ‘balancing’ of random events lends to the gambler’s fallacy. Under this impression, we think "if something happened before, there must be a balance in what will happen after" – of course this is not the case!
There is no equilibrium force in randomness. The law of large numbers isn’t a law of equilibrium, it is a law of dilution.
Most of us would probably catch this issue if presented with a well-framed question about how random samples balance out as their sample sizes increase. But, when we remove the nice framing, we have a tendency to fall for the fallacy.
Amos Tversky and Daniel Khaneman performed multiple studies on cognitive biases. They often presented carefully crafted questions to scientists and statistians that exposed biases and fundamental misunderstandings – even among these professionals! Below is one question they posed that exposes vulnerability to the gambler’s fallacy. Check it out and see how you would answer!
"The mean IQ of the population of eighth graders in a city is known to be 100. You have selected a random sample of 50 children for a study of educational acheivements. The first child tested has an IQ of 150. What do you expect the mean IQ to be of the whole sample?¹" – Tversky & Kahneman
The correct answer is 101, we’ll prove it below. Tversky and Kahneman report that a "surprisingly large number of people" get this wrong. People think that the answer is 100 because the other 49 samples will offset the abnormally large IQ in the first sample – this simply isn’t the case. The other samples are independent of the first. This is an example of the gambler’s fallacy outside of the casino!
Just in case you don’t believe me, below is the expected value calculation of a sample of the 50 students, given that one of the observations has an IQ of 150.
They don’t offset each other, but they can dilute each other – let’s see what happens if our sample was increased to 100 students?
The more random samples, the closer the expected value of the sample gets to 100 – even with the extremely high IQ student in our sample. This convergence is driven by dilution, not equilibrium!
We conflate the probability of a series of random events with a single random event
Here are two similar questions with very different answers:
What is the probability of flipping 10 heads in a row with a fair coin?
A fair coin has been flipped 9 times, all 9 flips were heads. What is the probability that the 10th flip is heads as well?
Can you spot the difference? One is asking about the probability of a series of events; the other has tricky framing, but is only asking about the probability of a single event.
The first question depicts a pretty unlikely event. We can go beyond intuition and calculate the exact probability using the binomial distribution, which simplfies to: 0.5¹⁰ = 0.1%- I would call that ‘unlikely.’
The second question asks; given that a pretty unlikely event has occured, what is the probability that another, independent event takes place. Since we are only talking about the 10th flip, the answer is simply 50%. But we feel like it is less because a strange, but independent event preceeded it.
If we are not disciplined in separating these two questions (probability of a series of events vs. probability of a single event) in our minds, we can be suciptable to the gambler’s fallacy.
Let’s go back to the roulette table to finish this point. Imagine you lost on black 10 times in a row – you feel that there is no way you could lose 11 times in a row. You have to win the next round! It is true that the probability of losing 11 times in a row is very low. But, that probability is only applicable when you sit down at the table before any spins. Once you’ve lost 10 in a row, the probability that you lose the 11th time is simply the probability that you lose in general. Nothing that happend before matters!
3 – What problems can the gambler’s fallacy cause in your work as a data scientist? How can you avoid them?
I’ve curated a list of three areas of Data Science that I think are most likely to be impacted by the gambler’s fallacy. Similar to the last section, my goal here is not to make a comprehensive list, but to highlight what I think are the most common areas that could be impacted.
- The problem: Our tendency towards the gambler’s fallacy can make us inclined to accept trends or patterns too quickly or with too small of a sample size. The fallacy causes us to feel like fewer random variables (i.e., smaller random samples) are needed to observe signal from the noise. How to Avoid it: This is avoided by using statistical techniques to calculate the Probability that the patterns we observe in samples are by chance.
- The problem: The gambler’s fallacy can cause us to design tests with sample sizes that are too small. This causes our tests to be under powered, meaning the probability that we are able to pick up on a trend/pattern from our experiment is too low. How to avoid it: We can avoid this problem by calculating the power of a test and deciding if we are okay with it. If we simply use our intuition to estimate a good sample size – we are quite likely to select too small of a sample. The statistical calculation of power removes the need for us to use intuition in the decision.
- The problem: The gambler’s fallacy can make us too conservative in accepting replicated results. Since the gambler’s fallacy compels us to believe that small samples are representative of the population and other small samples (Tversky & Kahneman), we have a bias to expect too much similarity between replications of tests and experiments. For example, let’s say we are testing out a new fertilizer on 15 bean plants. We get a result that show that the fertilizer helps bean plants grow with a p-value of less than 5%. We decide to replicate the experiment a little later and get a statistically insigficant p-value of 15%, but a positive average estimate of growth from the fertilizer. Given that the second experiment is ‘insignificant’, we have a bias to say that the results were not replicated – even if we have a small sample size in both of the experiments. How to avoid it: Once again, the best way to avoid this problem is through Statistics and calculations. The ‘replication probability’ can be calculated using key statistics from the original experiment/test, such as the effect size, the standard error and the statistical power. With these metrics and the sample size of the replication, we can get a quantitative (rather than intuitive) understanding of how ‘hard’ it will be to replicate the results. In the bean plant example, if we found that there was a 50% chance that the results would be replicated with a sample of 15 bean plants, we wouldn’t be too quick to discard our findings from the first experiment because the second one didn’t replicate the statistical significance.
Conclusion
We have covered the definition of the gambler’s fallacy, why it is appealing to us and how it can cause problems in our work as data scientists. The gambler’s fallacy can show up in multiple places outside of the casino and causes problems wherever we find it. The best way to combat against the fallacy is by relying on statistics and calculations rather than our intuition to make decisions and conclusions.
- Tversky, A., & Kahneman, D. (1971). Belief in the law of small numbers. Psychological Bulletin, 76(2), 105–110. https://doi.org/10.1037/h0031322
