# Fooled By Randomness

In my last post I discussed the myth of the SI jinx. Here is a brief recap. Athletes and teams are usually on the cover of SI for extraordinary performances. Almost always, an extraordinary performance is followed by an ordinary performance. So the SI jinx is easily explained by athletes and teams that have regressed back to their normal performance level. Contrary to popular belief, therefore, Sport Illustrated is not actually causing an athlete or team to play worse. However, people have a difficult time understanding this, and this is largely due to their inability to perceive and understand randomness. In this post, I want to build off of this point by illustrating just how bad we are with randomness. Let’s start with ipods.

When Apple first sold the ipod shuffle, users complained that it was not random enough. Though the engineers at Apple had programmed the ipod shuffles to be random, people were convinced that they were not. The problem was that “the randomness didn’t appear random, since some songs were occasionally repeated.” I took to the Apple blogosphere to see if this was true and on the Google’s first hit I found the following two posts:

User 1: There are 2800 songs in my ipod, I found that the Shuffle Songs function is not random enough, it always picks up the songs which I had played in the last one or two days.

User 2: It is random, which is why it’s not paying attention to whether or not you’ve played the songs lately.

User 2 is right, the ipod shuffle *is* random, making it entirely possible for a song to be played two days in a row, or two times in a row for that matter. The mistake made by User 1, is that people perceive streaks and patterns as indications that sequences are not random, *even though random sequences inherently contain streaks and patterns.*

Our tendency to misinterpret randomness is exemplified by the gambler’s fallacy, which describes our intuition’s habit of believing that the odds of something with a fixed probability are influenced by recent occurrences. For example, we think that the more times a coin lands on heads the more chances it has of landing on tails. In reality though, if a coin landed on heads one hundred times in a row it would still have a 50/50 chance of landing on heads the 101st time.

We make the same mistake when we watch sports. In 1985 Cornell psychologist Thomas Gilovich published a paper that “investigated the origin and the validity of common beliefs regarding the ‘hot hand’ and ‘streak shooting’ in the game of basketball.” His study was motivated by the common belief shared by fans, coaches, and players that a player’s chance of hitting a shot are greater following a hit as opposed to a miss. To see if basketball players actually “heat up,” Gilovich collected shooting stats from the Philadelphia 76ers 1980-81 season. He found that the chance a basketball player has of making a shot is actually unrelated to the outcome of his previous shot. In his words:

Contrary to the expectations expressed by our sample of fans, players were not more likely to make a shot after making their last one, two, or three shots than after missing their last one, two, or three shots. In fact, there was a slight tendency for players to shoot better after missing their last shot… the data flatly contradicts the notion that “success breeds success” in basketball and that hits tend to follow hits and misses tend to follow misses (1991, p. 12).

Gilovich’s conclusion comes as a surprise to most people. For some reason, our intuition tells us that a basketball player’s field goal percentage *is* influenced by his previous shots. This is why we want a player who is shooting well to continue to shoot, and vice versa.

Similar results have been found with baseball players and baseball teams. Michigan State University psychologist Gordon Wood demonstrated that the probability of an MLB team winning after a win, or losing after a loss, was fifty percent after analyzing the outcomes of all 1988 Major League Baseball games (26 teams & 160 games). Likewise, Indiana University statistician Christian Albright found the same with batters. He states that, “The behavior of all players examined… does not differ significantly from what would be expected under a model of randomness.” Like the outcome of a basketball shot, an MLB game and at bat were unaffected by past performance

None of these studies are denying that streaks exist; but they are saying that our intuition does a poor job of understanding and perceiving randomness – we mistakenly “see” patterns amongst randomness.

There are powers and perils to this cognitive bias. If you bet your life savings on a falsely perceived streak in the stock market, you could easily lose a life’s savings. Likewise in gambling, if you have gotten lucky on a slot machine you will want to keep going thinking that you have found a “hot” slot (in the end, of course, you will most likely have less than you started). On the other hand, our tendency to see order amongst random-chance events is an incredibly useful survival technique. Think what it would be like if you perceived the world as a series of random events; imagine that headache. With this in mind (not the headache), it seems awfully useful that we can “see” patterns that aren’t actually there.

- Thanks Nassim Taleb for the title of the post.
- The ipod shuffle discussion can be found here.

More baseball related evidence can be found in Madison Bumgarner’s recent pitching. After giving up 8 runs off 9 hits in 1/3 of an inning, the San Francisco Giant was pulled. In his next start, the 21 year old came back to pitch 7 innings, giving up one run (on a blown call), and striking out 11. Why was this surprising?

Because we have a hard time understanding randomness.

Fantastic post, I look forward to reading more in the future.

Likewise with the Twins. They won 14 of 16 in the first half of June, then followed that up with a five game loosing streak. They’re hot, they’re cold? I think not.

About the iPod “shuffle”: I think there are two different types of randomness being confused here. If you “shuffle” a deck of cards and then deal them out, you will _not_ get the Ace of Spades followed 5 cards later by the Ace of Spades.* If, on the other hand, you do a random selection with replacement, you may get the behavior being complained about. It sounds like Apple used a word that has a common, everyday meaning to denote something with a completely different behavior.

[*pedantic note: That is assuming that the first appearance of the Ace is not just before the deck runs out before reshuffling and dealing again. In this case, one could end up with Aces appearing quite close together.]

I think Apple did use shuffle in the same way that it is used in regard to playing cards. But they also advertised it as being random – and that is where my beef comes in.

“…the gambler’s fallacy, which describes our intuition’s habit of believing that the odds of something with a fixed probability are influenced by recent occurrences. For example, we think that the more times a coin lands on heads the more chances it has of landing on tails. In reality though, if a coin landed on heads one hundred times in a row it would still have a 50/50 chance of landing on heads the 101st time.”

Here, I think, people confuse different kinds of randomness, rather than misunderstand randomness, per se. The ‘randomness’ of an occurrence is based on the statistical probability that an event will or (or will not) occur. In the case of a coin toss, only one of two possible outcomes can occur for each flip (assuming the coin is not, by accidental or intentional design, ‘biased’ to land one way rather than the other).

The first statistical probability is; prior to recording the results of a single coin flip, what is the chance a coin will land ‘the same side up’ 101 times in a row during the first 101 coin flips, and not 101 times in a row at some point during a series of 101 + n (number of) successive coin flips?

I have absolutely no idea how to calculate the probability a coin flip will result in a ‘same way landing’ 101 times in a row during its first 101 flips, but it isn’t 50/50.

The second probability is; having flipped a coin 100 times and knowing the coin landed the same side up 100 times in a row, what is the chance of the coin landing the same way after the next flip? This time the answer is 50/50.