Extended Subset

Steve and I have developed some big new ideas!

db71e8d0992925ce2dd01c5918080d1b07532761

We’ll be going public with it at some point.

From Wikipedia:

1.  A (random) family has two children, and one of the two children is a boy named Jacob. What is the probability that the other child is a girl?

For these types of questions usually we are expected to make some standard simplifying assumptions: every child is either a boy or a girl, boys and girls are equally likely, and each one is completely random and independent of prior children. I.e., sex is a fair coin.

We are also usually taught that when probabilistic events are independent we should rely on the principle that “the next” coin flip will revert to its built-in probability of coming up tails. The trick to this is learning to ignore extraneous information such as the fact that the coin has come up head the previous nine times in a row.

So when I read that question the first time I, like most people I’ve tried it on, quickly concluded that p_girl = p_boy = 0.5.

I was shocked to learn that the probability of the other child being a girl depends on how common the name Jacob is in the population.

I didn’t believe it, so I made up a spreadsheet to play with and worked it out precisely, and I still don’t really believe it. If Jacob is a very uncommon name, p_girl approximates 0.5 as expected. But if Jacob is in fact a very common name, p_girl approaches 0.6667!

How can this be the case? The WP article explains it pretty well, so I won’t attempt to explain it fully here (plus I’m still a little upset about it). I’ll attach my spreadsheet for you to play with/poke holes in. I thought it might rely on the assumption that two children in the same family would not be given the same name, but it doesn’t.

Why is our intuition so wrong? How can we fix our intuition, or at least develop an awareness for when we are on shaky ground?  It seems that there is a great appeal to force-fitting the problem to a simplified model which we can reason about.  Perhaps there is some subconscious reasoning going on here: If reality approximates our model, then we can reason about it approximately. On the other hand, in cases where we lack a sufficient model with which to reason about reality then there is no reason to believe that reasoning with the wrong model is going to systematically turn out worse than not reasoning with no model!

I think the primary bit of force-fitting most of us do in this case is to map the question onto the simpler and much easier to solve question about discrete probabilistic events. After all, in real life children are generally sequential and largely independent discrete events. Consider how close the first question is to this one:

2.  An (arbitrary) person flipped a (fair) quarter twice, and the first time it came up heads showing the year 2006. What is the probability that the second toss was tails?

Surely to this question the answer is p = 0.5. Correct me if I’m wrong.

So how does (2) differ from (1)? Here are a few differences I can spot:

1. Problem (1) selects a (random) family of two from a population of many families. Problem (2) describes two independent events.
2. Problem (1) gives information about one of two children without assigning an ordering to them, Jacob may have been the first child or the second child. In (2) the coin tosses are sequential.
3. Problem (2) gives us information about the “first” event. Problem (1) just tells us about “one” of the children.

Can we generalize any principles that will help us recognize when our intuition is crossing that invisible line into error?  I can’t come up with much here, readers please offer some ideas! But perhaps is enough for now: One must be extremely careful about inadvertently shifting between discrete event and population selection models.

Next time: How this relates computer security to special relativity.

Well it’s taken me far too long to start blogging my random thoughts. The world seems to have waited patiently.

So here it is - my initial post:

c4be06692ff77ae2ce6759ccc480b3df2961f8c3

Do you like it? Isn’t it wonderful?

Just one problem…

It’s only the SHA-1 hash of a rough summary of what I would have liked to put as my first post.  Releasing the info publicly right now would cause a major inconvenience for a lot of folks.

Nevertheless, I intend on breaking the story right here, just as soon as is practical. Stay tuned.