June 29, 2007
Evolution-5: How probability intuition can lead us astray
One of life's ironies is that the difficulty in understanding the mathematics of Darwin's theory of natural selection may actually be caused by natural selection itself.
As we saw earlier, natural selection does not try for maximum benefit but instead works on a 'just good enough for now' principle. Steven Pinker in his book How the Mind Works (1997) is a cognitive scientist who believes that natural selection has been the driver for most aspects of our bodies and our behavior, and that the brain, being just another organ, has evolved to do what it does to effectively meet the challenges it faced at various times in our somewhat distant past. Pinker points out that humans, when compared with other animals, have unusually large brains compared to body size but that this rapid expansion in brain size occurred more than 100,000 years (or about 5,000 generations) ago (Pinker, p. 198) and then leveled off after that. This means that the structure of our present brains has been largely determined by a time when humans were hunter-gatherers and foragers.
This means that although modern life is undoubtedly very complex and require us to meet a vast array of challenges, our brains are best suited to meet the challenges of our ancient forebears, not those of driving on a highway or learning to operate a computer or solving sudoku puzzles. Thus we are very good at identifying faces and shapes, seeing things in depth, reacting to predatory dangers, and acting on instincts such as ducking when an object is thrown at our heads, etc, because our brains have probably evolved modules that handle such things efficiently. But we are not so good at solving quadratic equations. The kind of mathematics that helped our hunter-gatherer ancestors survive did not require much beyond an elementary sense of number. As for probability, simple concepts largely based on induction and extrapolating from past experiences, are sufficient.
But as culture developed in the last 10,000 years with the advent of more settled agrarian societies and written language, we now find ourselves having to struggle a bit to master the concepts needed to face today's challenges. They do not come 'naturally' to us, by which I mean that there are no brain modules that have evolved to enable us to quickly grasp and understand and respond to them.
This is especially true of probability and statistics. There was no need for our ancestors to develop modules to make Bayes' Theorem or the Central Limit Theorem easily understandable, which explains why our intuitions are so often led astray. For example, many people fall prey and lose money because of the 'gambler's fallacy' because they put their faith in a spurious 'law of averages', believing that the more repeated occurrences you have of the same thing (say getting heads on a coin toss or coming up black in a roulette wheel), the more likely a different outcome becomes on the next play. Similarly people who play the lottery numbers tend to avoid numbers that have won recently.
While mathematical sophisticates may look down on such naïvete , Pinker points out that such expectations are perfectly consistent with the kinds of probability experiences our hunter-gatherer ancestors experienced and which we still experience in most everyday life. After several days of rain, a dry day is more likely. After seeing several elephants appear in a line, it was more likely not to see one. In fact, event repetitions that are finite and terminate and change are the norm in nature, not the exception. Hence believing such things and acting upon such beliefs has some survival value that makes it plausible that our brains evolved modules that encoded those expectations, making us instinctively sympathetic towards believing things like the gambler's fallacy.
The reason that so many are fooled by the gambler's fallacy is that the creators of the gambling devices go to great lengths to make each event independent of the previous ones, thus violating our natural expectations. We thus have to consciously learn to sometimes go against our 'natural' instincts and this takes effort and is not easy.
Even though I consider myself fairly adept at mathematical manipulations, I am often humbled by how easily my intuition is led astray when confronted with a novel statistics problem. Take for example this case, which may be familiar to people who have taken an elementary statistics course, but fooled me when I first encountered it.
Suppose the incidence of some disease is fairly rare in a population, say about one in a thousand. You are told that there is a test for this disease that is pretty good in that it that has a 'false positive' rate of only 5%, meaning that if a randomly selected group is tested, only 5% of the people who do not have the disease will have test results that come out positive. Also you are told that the false negative rate is zero, meaning that if someone does have the disease, the test will definitely come out positive.
Suppose you are among those who are part of this random testing. To your dismay, the result is positive. What do you think are your chances of actually having the disease?
Most people would think that it is very high. They may put it as high as 95%, thinking that if there is a 5% false positive rate and 0% false negative rate, that means that the likelihood of someone testing positive having the disease is 95%. This sounds eminently reasonable.
But the actual chance of you having the disease despite testing positive is just 1 in 51 or less than 2%! How come? This becomes easier to understand if we shift from talking in terms of probabilities (which I have pointed out are not so intuitive) to talking about numbers. Suppose you are one of 1000 people being randomly tested. (Any size will do. I have chosen 1000 because it is a nice round number.) Then an incidence of 1 in 1000 means that we expect only one person to actually have the disease (and who will test positive), and 999 to be free of the disease. But a 5% false positive rate will result in about 50 of the 999 people who do not have the disease also testing positive. So your chance of actually having the disease is the chance that you happen to be that one person with the disease out of the 51 testing positive.
What the positive test result has done is provide a twenty-fold increase in the odds of your having the disease from 1 in 1000 (or 0.1%) to 1 in 51 (or slightly less than 2%), but your chances are still extremely good (over 98%) of not having the disease. I suspect a lot of people get unduly terrified by test results of this kind because doctors may not know how to present the data in ways that give them a better sense of estimating the probability. (Of course, I am assuming that you were selected randomly for this test. If the doctor recommended that you get the test because you had other symptoms that caused her to suspect you had the disease, then that would further increase the odds of you having the disease.)
The lesson here is to be wary of our 'gut' feelings when dealing with certain mathematics concepts, especially involving probability and statistics. This may partially explain why Darwin's theory of natural selection, dealing as it does with small probabilities and long time scales, is so hard for many to digest because they are outside the range of things we experience on a daily basis. In future postings, I will look at some of the issues that come up.
POST SCRIPT: Sicko opening nationwide on Friday
Michael Moore's new documentary on the health care system Sicko will be at the Cedar-Lee (2163 Lee Rd) in Cleveland Heights starting on Friday, June 29, 2007. The show times are noon, 2:30, 5:00, 7:30, and 10:00 but you should check before you go.
Moore also appeared on The Daily Show to point out once again what a scandal the health care system in the US is, where it is actually in the interests of the profit-driven health insurance companies to deny health care to patients.