There has been tremendous confusion here and at Uncommon Descent about what I’ll call the ‘all-heads paradox’.
The paradox, briefly stated:
If you flip an apparently fair coin 500 times and get all heads, you immediately become suspicious. On the other hand, if you flip an apparently fair coin 500 times and get a random-looking sequence, you don’t become suspicious. The probability of getting all heads is identical to the probability of getting that random-looking sequence, so why are you suspicious in one case but not the other?
In this post I explain how I resolve the paradox. Lizzie makes a similar argument in her post Getting from Fisher to Bayes, but there are some differences, so keep reading.
Early in the debate, I introduced a thought experiment involving Social Security numbers (link, link). Imagine you take an apparently fair ten-sided die labeled with the digits 0-9. You roll it nine times and record the number you get, digit by digit. It matches your Social Security number. Are you surprised and suspicious? You bet.
As I put it then:
My point is that for anyone to roll their SSN (or another personally significant 9-digit number) is highly unlikely, because the number of personally significant 9-digit numbers is low for each of us while the space of possible 9-digit numbers is large.
If someone sits down and rolls their SSN, then either:
1. The die was fair, the rolls were random, and they just got lucky. It’s pure coincidence.
2. Something else is going on.
Just as my own SSN is significant to me, 500 heads in a row are significant to almost everybody. So when we express suprise at rolling our SSN or getting 500 heads in a row, what we are really saying is “It’s extremely unlikely that this random number (or head/tail sequence) would just happen to match one of the few that are personally significant to me, instead of being one of the enormous number that have no special significance. Hmmm, maybe there’s something else going on here.”
That helps, but it doesn’t seem to completely resolve the paradox. It still seems arbitrary and subjective to divide the 9-digit numbers into two categories, “significant to me” and “not significant to me”. Why am I suspicious when my own SSN comes up, but not when the SSN of Delbert Stevens of Osceola, Arkansas comes up? His SSN is just as unlikely as mine. Why doesn’t it make me just as suspicious?
In fact, there are millions of different ways to carve up the 9-digit numbers into two sets, one huge and one tiny. Should we always be surprised when we get a number that belongs to a tiny set? No, because every number belongs to some tiny set, properly defined. So when I’m surprised to get my own SSN, it can’t be merely because my SSN belongs to a tiny set. Every number does.
The answer, I think, is this: when we roll Delbert’s SSN, we don’t actually conclude that the die was fair and that the rolls were random. For all we know, we could roll the die again and get Delbert’s number a second time. The outcome might be rigged to always give Delbert’s number.
What we really conclude when we roll Delbert’s number is that we have no way of determining whether the outcome was rigged. In a one-off experiment, there is no way for us to tell the difference between getting Delbert’s (or any other random person’s) SSN by chance versus by design or some other causal mechanism. On the other hand, rolling our own SSN does give us a reason to be suspicious, precisely because our SSN already belongs to the tiny set of “numbers that are meaningful to me”.
In other words, when we roll our own SSN, we rightly think “seems non-random”. When we roll someone else’s SSN, we think (or should think) “no particular reason to think this was non-random, though it might be.”
The same reasoning applies to the original “all-heads paradox”.