The Science of Discworld Revised Edition

The reason for this is an extremely important factor – ‘selective reporting’, which is a type of narrativium in action. This factor tends to be ignored in most conventional statistics. That perfect hand at bridge, for instance, is far more likely to make it to the local or even national press than an imperfect one. How often do you see the headline B RIDGE PLAYER GETS ENTIRELY ORDINARY HAND , for instance? The human brain is an irrepressible pattern-seeking device, and it seizes on certain events that it considers significant, whether or not they really are. In so doing, it ignores all the ‘neighbouring’ events that would help it judge how likely or unlikely the perceived coincidence actually is.
    Selective reporting affects the significance of those Formula One times. If it hadn’t been them, maybe the tennis scores in the US Open would have contained some unusual pattern, or the football results, or the golf … Any one of those would have been reported, too – but none of the failed coincidences, the ones that
didn’t
happen, would have hit the headlines. F ORMULA O NE DRIVERS RECORD DIFFERENT LAP TIMES … If we include just ten major sporting events in our list of would-be’s that weren’t, that one in ten thousand chance comes down to only one in a thousand.
    Having understood this, let’s go back to the Israeli fighter pilots. Conventional statistics would set up the obvious sample space, assign probabilities to boy and girl children, and calculate the chance of getting 84% girls in a purely random trial. If this were less than one in a hundred, say, then the data would be declared ‘significant at the 99% level’. But this analysis ignores selective reporting. Why did we look at the sexes of Israeli fighter pilots’ children in the first place? Because our attention had
already
been drawn to a clump. If instead the clump had been the heights of the children of Israeli aircraft manufacturers, or the musical abilities of the wives of Israeli air traffic controllers, then our clump-seeking brains would again have drawn the fact to our attention. So our computation of the significance level tacitly excludes many other factors that
didn’t
clump – making it fallacious.
    The human brain filters vast quantities of data, seeking things that appear unusual, and only then does it send out a conscious signal:
Wow! Look at that!
The wider we cast our pattern-seeking net, the more likely it is to catch a clump. For this reason, it’s illegitimate to include the data that brought the clump to our attention as part of the evidence that the same clump is unusual. It would be like sorting through a pack of cards until you found the ace of spades, putting it on the table, and then claiming miraculous powers that unerringly accomplish a feat whose probability is one in 52.
    Exactly this error was made in early experiments on extra-sensory perception. Thousands of subjects were asked to guess cards from a special pack of five symbols. Anyone whose success rate was above average was invited back, while the others were sent home. After this had gone on for several weeks, the survivors all had an amazing record of success! Then these ‘good guessers’ were tested some more. Strangely, as time went on, their success rate slowly dropped back towards the average, as if their powers were ‘running down’. Actually, that effect wasn’t strange at all. It happened because the initial high scores were included in the running total. If they had been omitted, then the scoring rate would have dropped, immediately, to near average.
    So it is with the fighter pilots. The curious figures that drew researchers’ attention to these particular effects may well have been the result of selective reporting, or selective attention. If so, then we can make a simple prediction: ‘From now on, the figures will revert to fifty-fifty.’ If this prediction
fails
, and if the results instead confirm the bias that revealed the clump, then the new data can be considered significant, and a significance level can sensibly be assigned by the usual methods. But the smart money is on a fifty-fifty split.
    The alleged decline in the human sperm count may be an example of selective reporting. The story, widely repeated in the press, is that over the past 50 years the human sperm count for ‘normal’ men has halved. We don’t mean selective reporting by the people who published the first evidence – they took pains to avoid all the