Significance Bias
Have you ever noticed that your phone rings every time you’re in the shower? You end up missing the call and say something along the lines of “why does this always happen?!”
Phone Rings in the Shower
But how often does that actually happen? Actually not very often, but we perceive that it does. This is something I like to call significance bias. The only significant event is your phone ringing while you’re in the shower therefore you only remember this event. Thus leading to you thinking it happens all the time.
There are four possible events that can happen. The first is, that your phone doesn’t ring and you don’t take a shower. Nothing happened. Not significant.
Next, you take a shower and your phone doesn’t ring. Very common, most people take a shower every day and your phone doesn’t ring during that time. Not significant.
Thirdly, your phone rings and you’re not taking a shower. This is also very common, people call our phones all the time. Especially about our car’s extended warranty. Again not significant.
Finally, your phone rings and you’re in the shower. That is the only event that is significant since you can’t answer the phone and be in the shower at the same time. You remember this event. After this event happens a few times you begin to think it is very common. But, in fact, compared to the other events that can happen it is quite uncommon.
Random Drug Testing
On Friday, August 19th Saints punter, Blake Gillikin, kicked an 81-yard punt in one of the 2022 preseason football games. An absolutely insane punt. Blake on that Sunday tweeted out “Punt at your own risk” after being hit with a random drug test by the NFL.
Many sports newsgroups have reported this event and put “random”, like this in quotes, in their headline. This is implying that the drug test wasn’t random but instead Gillikin was targeted for the drug test after his great performance in the game a few days prior. This isn’t the only time speculation of random drug tests wasn’t so random because they came after a player has a really good game.
Using the Significance Bias Model
But let’s use the significance bias model for this scenario. First, a player doesn’t have a good game and doesn’t get drug tested. Nothing happened, not significant.
A player plays a good game and doesn’t get drug tested. Happens all the time, many players play great games each week and don’t get tested. Not a significant event.
Next, a player doesn’t have a good game and gets drug tested. Again very common, as there are 12,000 drug tests a year for NFL players, and 10 random players from each team are tested each week. You’re guaranteed to drug test someone who had a bad game that week. Again not significant.
Finally, a player has a good game and gets drug tested. Which is what happened here. It is the only suspicious event that can happen. It’s the only newsworthy event. It is significant enough that we notice this outcome yet not the other three outcomes.
According to the NFL Drug testing Policy, every player is tested at least once a year. Players can be tested up to six times a year. So although drug testing is random there are still restrictions on the randomness. To imply that drug testing is not random because players get tested after a good game is just wrong.
The significance bias is quite common and can be seen in many aspects of your life. But before you think an outcome is oddly suspicious, just think if the other possible outcomes are just as noteworthy. If not, it may be a case of significance bias.
Pingback: Why it is Difficult for People to Understand Statistics - ryantvackner
Pingback: Life is a Prisoner's Dilemma - ryantvackner