Sampling is a
respectable
way of estimating something that we cannot count directly.
Richard-Dawkins-Unweaving-the-Rainbow
Actually, the trick here is so obvious it probably wouldn't fool many people.
But how about the following?
A well-known 'psychic' goes on television, a lucrative engagement fixed up over lunch by his publicity agent. Staring out of ten million screens with hypnotically smouldering eyes (nice job by Make-up and Lighting), our imaginary seer intones that he feels a strange, spiritual rapport, a vibrating resonance of cosmic energy, with certain members of his audience. They will be able to tell who they are because, even as he utters his mystic incantation, their watches will stop. After only a brief pause, a telephone on his table rings and an amplified voice in awed tones announces that its owner's watch stopped dead within seconds of the clairvoyant's words. The caller adds that she had a premonition that this was going to happen even before she looked down at her watch, for something in her hero's burning eyes seemed to speak directly to her soul. She felt the 'vibrations' of 'energy'. Even as she is speaking, a second telephone rings. Yet another watch has stopped.
A third caller's grandfather clock stopped - surely a weightier feat than stopping a little watch whose delicate hairspring would naturally be more susceptible to psychic forces than the massive pendulum of the grandfather! Another viewer's watch actually stopped a little before the celebrated mystic made his pronouncement - is this not an even more impressive feat of psychic control? Yet another watch has been more impatiently susceptible to occult forces. It had stopped a whole day before, at the very moment when its owner looked at the famous mystic's photograph in the newspaper. The studio audience gasps its appreciation. This, surely, is psychic power beyond all scepticism, for it happened a whole day early! 'There are more things in heaven and earth, Horatio . . . '
What we need is less gasping and more thinking. This chapter is about how to take the sting out of coincidence by quietly sitting down and calculating the likelihood that it would have happened anyway. In the course of this, we shall discover that to disarm apparently uncanny coincidences is more interesting than gasping over them anyway.
Sometimes the calculation is easy. In a previous book I gave away the number of the combination lock on my bicycle. I felt safe in doing so because obviously my books would never be read by the kind of person who would steal a bicycle. Unfortunately somebody did steal it, and I now have a new lock with a new number, 4167. I find this number easy to remember. 41 is imprinted in my memory as the arbitrary code used to identify my clothes and shoes at boarding school. 67 is the age at which I am due to retire. Obviously there is no interesting coincidence here: whatever the number had been, I'd have searched my life for a mnemonic recipe and I'd have found it. But mark the sequel. On the day of writing this, I received from my Oxford college a letter saying:
Each person authorized to use the photocopiers is issued with a personal code number which permits access. Your new number is 4167.
My first thought was that I'd undoubtedly lose this piece of paper (I quickly lost its equivalent last year) and I must immediately think of a formula to fix it in my memory. Something similar to the mnemonic by which I remember my bicycle combination, perhaps? So I looked again at the number on the letter and, to borrow a neat line from Fred Hoyle's science fiction novel The Black Cloud, the figures on the piece of paper seemed to swell to a gigantic size. I didn't need a new mnemonic. The number was identical. I rushed to tell my wife of the amazing coincidence, but on more sober reflection I shouldn't have bothered.
The odds of this happening by chance alone are easily calculated. The first digit could have been anything from 0 to 9. So there is a one in 10 chance of getting a 4 and matching the bicycle lock. For each of these ten possibilities, the second digit could have been anything from 0 to 9, so again there is a one in 10 chance of matching the bike lock's second dial. The odds of matching the first two digits is therefore one in 100 and, following the logic through the other two digits, the odds of matching all four digits of the bicycle lock is one in 10,000. It is this large number that is our protection against theft.
The coincidence is impressive. But what should we conclude? Has something mysterious and providential been going on? Have guardian angels been at work behind the scenes? Have lucky stars swum into Uranus? No. There is no reason to suspect anything more than simple accident. The number of people in the world is so large compared with 10,000 that somebody, at this very moment, is bound to be experiencing a coincidence at least as startling as mine. It just happens that today was my day to notice such a coincidence. It isn't even an added coincidence that it happened to me on this particular day, while I was writing this chapter. I had in fact written the first draft of the chapter some weeks ago. I reopened it today, after the coincidence occurred, in order to insert this anecdote. I shall surely reopen it many times to revise and polish, and I shall not remove the references to 'today': they were accurate when written. This is another way in which we habitually inflate the impressiveness of coincidence in order to make a good story.
We can do a similar calculation for the television guru whose psychic miasma seemed to stop people's watches, but we'll have to use estimates rather than exact figures. Any given watch has a certain low probability of stopping at any moment. I don't know what this probability is, but here's the kind of way in which we could come to an estimate. If we take just digital watches, their battery typically runs out within a year. Approximately, then, a digital watch stops once per year. Presumably clockwork watches stop more often because people forget to wind them
and presumably digital watches stop less often because people sometimes remember to renew the battery ahead of time. But both kinds of watches probably stop as often again because they develop faults of one kind or another. So, let our estimate be that any given watch is likely to stop about once a year. It doesn't matter too much how accurate our estimate is. The principle will remain.
If somebody's watch stopped three weeks after the spell was cast, even the most credulous would prefer to put it down to chance. We need to decide how large a delay would have been judged by the audience as sufficiently simultaneous with the psychic's announcement to impress. About five minutes is certainly safe, especially since he can keep talking to each caller for a few minutes before the next call ceases to seem roughly simultaneous. There are about 100,000 five-minute periods in a year. The probability that any given watch, say mine, will stop in a designated five-minute period is about 1 in 100,000. Low odds, but there are 10 million people watching the show. If only half of them are wearing watches, we could expect about 25 of those watches to stop in any given minute. If only a quarter of these ring in to the studio, that is 6 calls, more than enough to dumbfound a naive audience. Especially when you add in the calls from people whose watches stopped the day before, people whose watches didn't stop but whose grandfather clocks did, people who died of heart attacks and their bereaved relatives phoned in to say that their 'ticker' gave out, and so on. This kind of coincidence is celebrated in the delightfully sentimental old song, 'Grandfather's Clock:'
Ninety years without slumbering. Tick, lock, tick, tock.
His life seconds numbering. Tick, tock, tick, tock.
It stopped . . . short. . . never to go again When the old man died.
Richard Feynman, in a 1963 lecture published posthumously in 1998, tells the story of how his first wife died at 9. 22 in the evening and the clock in her room was later found to have stopped at exactly 9. 22. There are those who would revel in the apparent mystery of this coincidence and feel that Feynman has taken away something precious when he gives us a simple, rational explanation of the mystery. The clock was old and erratic and was in the habit of stopping if tilted out of the horizontal. Feynman himself frequently repaired it. When Mrs Feynman died, the
nurse's duty was to record the exact time of death. She moved over to the clock, but it was in dark shadow. In order to see it, she picked it up - and tilted its face towards the light . . . The clock stopped. Is Feynman really spoiling something beautiful when he tells us what is surely the true - and very simple - explanation? Not for my money. For me, he is affirming the elegance and beauty of an orderly universe in which clocks stop for reasons, not to titillate human sentimental fancy.
At this point, I want to invent a technical term, and I hope you'll forgive an acronym. PETWHAC stands for Population of Events That Would Have Appeared Coincidental. Population may seem an odd word, but it is the correct statistical term. I won't keep using capital letters because they stand so unattractively on the page. Somebody's watch stopping within ten seconds of the psychic's incantation obviously belongs within the petwhac, but so do many other events. Strictly speaking, the grandfather clock's stopping should not be included. The mystic did not claim that he could stop grandfather clocks. Yet when somebody's grandfather clock did stop, they immediately telephoned in because they were, if anything, even more impressed than they would have been if their watch had stopped. The odd misconception is fostered that the psychic is even more powerful since he didn't even bother to mention that he could stop grandfather clocks, too! Similarly, he said nothing about watches stopping the day before or grandfathers' tickers suffering cardiac arrests.
People feel that such unanticipated events belong in the petwhac. It looks to them as though occult forces must have been at work. But when you start to think like this, the petwhac becomes really quite large, and therein lies the catch. If your watch stopped exactly 24 hours earlier, you would not have to be unduly gullible to embrace this event within the petwhac. If somebody's watch stopped exactly seven minutes before the spell, this might impress some people because seven is an ancient mystic number. And the same presumably goes for seven hours, seven days . . . The larger the petwhac, the less we ought to be impressed by the coincidence when it comes. One of the devices of an effective trickster is to make people think exactly the opposite.
By the way, I deliberately chose a more impressive trick for my imaginary psychic than is actually done with watches on television. The more familiar feat is to start watches that have stopped. The television
audience is invited to get up and fetch, out of drawers or attics, watches that have broken down, and hold them while the psychic performs some incantation or does some hypnotic eye work. What is really going on is that the warmth of the hand melts oil that has coagulated and this starts the watch ticking, if only briefly. Even if this happens in only a small proportion of cases, this proportion, multiplied by the large audience, will generate a satisfactory number of dumbfounded telephone calls. Actually,
as Nicholas Humphrey explains in his admirable expose of supernaturalism Soul Searching (1995), it has been demonstrated that more than 50 per cent of broken watches start, at least momentarily, if they are held in the hand.
Here's another example of a coincidence, where it is clear how to calculate the odds. We shall use it to go on and see how odds are sensitive to changing the petwhac. I once had a girlfriend who had the same birth date (though not in the same year) as my previous girlfriend. She told a friend of hers who believed in astrology, and the friend triumphantly asked how I could possibly justify my scepticism in the face of such overwhelming evidence that I had unwittingly been brought together with two successive women on the basis of their 'stars'. Once again, let's just think it through quietly. It is easy to calculate the odds that two people, chosen entirely at random, will have the same birthday. There are 565 days in the year. Whatever the birthday of the first person, the chance that the second will have the same birthday is 1 in 365 (forgetting leap years). If we pair people off in any particular way, such as taking the successive women friends of any one man, the odds that they will share their birthday are 1 in 365. If we take ten million men (less than the population of Tokyo or Mexico City), this apparently uncanny coincidence will have happened to more than 27,000 of them!
Now let's think about the petwhac and see how the apparent coincidence becomes less impressive as it swells. There are many other ways in which we could pair people off and still end up noticing an apparent coincidence. Two successive girlfriends with the same surname, although unrelated, for instance. Two business partners with the same birthday would also come within the petwhac; or two people with the same birthday sitting next to one another on an aeroplane. Yet, in a well- loaded Boeing 747, the odds are actually better than 50 per cent that at least one pair of neighbours will share a birthday. We don't usually notice this because we don't look over each other's shoulders as we fill in those tedious immigration forms. But if we did, somebody on most flights would go away muttering darkly about occult forces.
The birthday coincidence is famously phrased in a more dramatic way. If you have a roomful of only 25 people, mathematicians can prove that the odds are just greater than 50 per cent that at least two of them will share the same birthday. Two readers of an earlier draft of the book asked me to justify this astonishing statement. It's easier to calculate the odds that there won't be any shared birthdays and subtract from one. Forget about leap years because they're more trouble than they're worth. Suppose I bet you that with 25 people in a room, at least two will share a birthday. You bet, for the sake of argument, that there'll be no shared birthdays. We're going to do the calculation by working up to 23 people gradually,
starting with just one person in the room, and adding people one at a time. If at any point a match is found, I've won the bet, we stop the game and don't bother to add any more people. If we get to 23 people and there's still no match so far, you win the bet.
When the room contains only the first person, whom we may as well call A, the chance of 'no match so far' is trivially 1 (365 out of 365 chances). Now add a second person, B. The chance of a match is now one in 365. So the odds of 'no match so far' when B has joined A in the room are 364/365. Now add a third person, C, There's a one in 365 chance that C matches A and a one in 365 chance that C matches B, therefore the chance that he matches neither A nor B is 363/365 (he can't match both, because we already know that A doesn't match B). To get the total odds
of 'no match so far' we have to take this 363/365 and multiply it by the odds against a match in the previous round (s), in this case by 364/365. The same reasoning applies when we add the fourth person, D. The total odds of 'no match so far' are now 364/365 X 363/365 X 362/365. And so on until all 23 people are in the room. Each new person adds a new term that we have to bring in to the running multiplication sum, to compute 'no match so far'.
If you multiply this out for 23 terms (you have to go on down to 343/365) the answer comes to about 0. 49. This is the chance that there will not be any shared birthdays in the room. So there's a slightly greater than even chance that at least one pair of individuals in a committee of 23 will share a birthday. Most people's intuition would encourage them to bet against such a coincidence. But they'd be wrong. It is this kind of intuitive error that in general bedevils our assessment of 'uncanny' coincidences.
Here's an actual coincidence where, although it is a little harder, we can make a stab at estimating the odds approximately. My wife once bought for her mother a beautiful antique watch with a pink face. When she got it home and peeled off the price label she was amazed to find, engraved on the back of the watch, her mother's own initials, M. A. B. Uncanny? Eerie? Spine-crawling? Arthur Koestler, the famous novelist, would have read much into it. So would C. G. Jung, the widely admired psychologist and inventor of the 'collective unconscious', who also believed that a bookcase or a knife might be induced by psychic forces to explode spontaneously with a loud report. My wife, who has more sense, merely thought the coincidence of initials remarkably convenient and sufficiently amusing to justify telling the story to me - and here I am now telling it to a wider audience.
So, what really are the odds against a coincidence of this magnitude? We can begin by calculating them in a naive way. There are 26 letters in the
alphabet. If your mother has three initials and you find a watch engraved with three letters at random, the odds that the two will coincide is 1/26 x 1/26 x 1/26, or one in 17,576. There are about 55 million people in Britain. If every one of them bought an antique engraved watch we'd expect more than 3,000 of them to gasp with amazement when they discovered that the watch already bore their mother's initials.
But the odds are actually better than this. Our naive calculation made the incorrect assumption that each letter has a probability of 1/26 of being somebody's initial. This is the average probability for the alphabet as a whole, but some letters, such as X and Z, have a smaller probability. Others, including M, A and B, are commoner: think how much more impressed we'd be if the coinciding initials had been X. Q. Z. We can improve our estimate of odds by sampling a telephone directory.
Sampling is a respectable way of estimating something that we cannot count directly. The London directory is a good place to sample because it is large and London happens to be where my wife bought the watch and where her mother lived. The London telephone directory contains about 85,060 column inches, or about 1. 34 column miles, of private citizens' names. Of these, about 8,110 column inches are devoted to the Bs. This means that about 9. 5 per cent of Londoners have a surname beginning with B - much more frequent than the figure for an average letter: 1/26, or 3. 8 per cent.
So, the probability that a randomly chosen Londoner would have a surname beginning with B is about 0. 095 (~ 9. 5 per cent). What about
the corresponding probabilities that the forenames will begin with M or A? It would take too long to count forename initials right through the telephone book, and there'd be no point since the telephone book is itself only a sample. The easiest thing to do is take a subsample where forename initials are conveniently arranged in alphabetical order. This is true of the listings within any one surname. I shall take the commonest surname in England - Smith - and look at what proportion of the Smiths are M. Smith and what proportion are A. Smith. It is a reasonable hope that this will be approximately representative of the probabilities of forename initials for Londoners generally. It turns out that there are rather more than 20 column yards of Smiths altogether. Of these, 0. 075 them (53. 6 column inches) are M. Smiths. The A. Smiths fill 75-4 column inches, representing 0. 102 of all the Smiths,
If you are a Londoner and you have three initials, therefore, the chances of your initials being M. A. B. in that order are approximately 0. 102 X 0. 073 0-095 or about 0. 0007. Since the population of Britain is 55 million, this should mean that about 38,000 of them have the initials M. A. B. , but only if everybody among those 55 million has three initials. Obviously not everybody does but, looking down the telephone directory
again, it seems that at least a majority do. If we make the conservative assumption that only half of British people have three initials, that still means that more than 19,000 British people have identical initials to my wife's mother. Any one of them could have bought that watch and gasped with astonishment at the coincidence. Our calculation has shown that there is no reason to gasp.
Indeed, when we think harder about the petwhac, we find that we have even less right to be impressed. M. A. B. were the initial letters of my wife's mother's maiden name. Her married initials of M. A. W. would have seemed just as impressive had they been found on the watch. Surnames beginning with W are nearly as common in the telephone book as those beginning with B. This consideration approximately doubles the petwhac, by doubling the number of people in the country who would have been deemed, by a coincidence hunter, to have 'the same initials' as my wife's mother.
Moreover, if somebody bought a watch and found it to be engraved not with her mother's initials but with her own, she might consider it an even greater coincidence and more worthy to be embraced within the (ever-growing) petwhac.
The late Arthur Koestler, as I have already mentioned, was a great enthusiast of coincidences. Among the stories that he recounts in The Roots of Coincidence (1972) are several originally collected by his hero, the Austrian biologist Paul Kammerer (famous for publishing a faked experiment purportedly demonstrating the 'inheritance of acquired characteristics' in the midwife toad). Here is a typical Kammerer story quoted by Koestler:
On September 18, 1916, my wife, while waiting for her turn in the consulting rooms of Prof. Dr J. v. H. , reads the magazine Die Kunst; she is impressed by some reproductions of pictures by a painter named Schwalbach, and makes a mental note to remember his name because she would like to see the originals. At that moment the door opens and the receptionist calls out to the patients: 'Is Frau Schwalbach here? She is wanted on the telephone. '
It probably isn't worth trying to estimate the odds against this coincidence, but we can at least write down some of the things that we'd need to know. 'At that moment the door opens' is a little vague. Did the door open one second after she made the mental note to look up Schwalbach's paintings or 20 minutes? How long could the interval have been, leaving her still impressed by the coincidence? The frequency of the name Schwalbach is obviously relevant: we'd be less impressed if it had been Schmidt or Strauss; more impressed if it had been Twistleton- Wykeham-Fiennes or Knatchbull-Huguesson. My local library doesn't
have the Vienna telephone book, but a quick look in another large Germanic telephone directory, the Berlin one, yields half a dozen Schwalbachs: the name is not particularly common, therefore, and it is understandable that the lady was impressed. But we need to think further about the size of the petwhac. Similar coincidences could have happened to people in other doctors' waiting rooms; and in dentists' waiting rooms, government offices and so on; and not just in Vienna but anywhere else. The quantity to keep bearing in mind is the number of opportunities for coincidence that would have been thought, if they had occurred, just as remarkable as the one that actually did occur.
Now let's take another kind of coincidence, where it is even harder to know how to start calculating odds. Consider the often-quoted experience of dreaming of an old acquaintance for the first time in years and then getting a letter from him, out of the blue, the next day. Or of learning that he died in the night. Or of learning that he didn't die in the night but his father did. Or that his father didn't die but won the football pools. See how the petwhac grows out of control when we relax our vigilance?
Often, these coincidence stories are gathered together from a large field. The correspondence columns of popular newspapers contain letters sent in by individual readers who would not have written but for the amazing coincidence that had happened to them. In order to decide whether we should be impressed, we need to know the circulation figure for the newspaper. If it is 4 million, it would be surprising if we did not read daily of some stunning coincidence, since a coincidence only has to happen to one of the 4 million in order for us to have a good chance of being told about it in the paper. It is hard to calculate the probability of a particular coincidence happening to one person, say a long-forgotten old friend dying during the night we happen to dream about him. But whatever this probability is, it is surely far greater than one in 4 million.
So, there really is no reason for us to be impressed when we read in the newspaper of a coincidence that has happened to one of the readers, or to somebody, somewhere in the world. This argument against being impressed is entirely valid. Nevertheless, there may be something lurking here that still bothers us. You may be happy to agree that, from the point of view of a reader of a mass-circulation newspaper, we have no right to be impressed at a coincidence that happens to another of the millions of readers of the same newspaper who bothers to write in. But it is much harder to shake the feeling of spine-chilled awe when the coincidence happens to you yourself. This is not just personal bias. One can make a serious case for it. The feeling occurs to almost everybody I meet; if you ask anybody at random, there is a good chance that they will have at least one pretty uncanny story of coincidence to relate. On the face of it,
this undermines the sceptic's point about newspaper stories having been culled from a millions-strong readership - a huge catchment of opportunity.
Actually it doesn't undermine it, for the following reason. Each one of us, though only a single person, none the less amounts to a very large population of opportunities for coincidence. Each ordinary day that you or I live through is an unbroken sequence of events, or incidents, any of which is potentially a coincidence. I am now looking at a picture on my wall of a deep-sea fish with a fascinatingly alien face. It is possible that, at this very moment, the telephone will ring and the caller will identify himself as a Mr Fish. I'm waiting . . .
The telephone didn't ring. My point is that, whatever you may be doing in any given minute of the day, there probably is some other event - a phone call, say - which, if it were to happen, would with hindsight be rated an eerie coincidence. There are so many minutes in every individual's lifetime that it would be quite surprising to find an individual who had never experienced a startling coincidence. During this particular minute, my thoughts have strayed to a schoolfellow called Haviland (I don't remember his Christian name, nor what he looked like) whom I haven't seen or thought of for 45 years. If, at this moment, an aeroplane manufactured by the de Haviland company were to fly past the window, I'd have a coincidence on my hands. In fact I have to report that no such plane has been forthcoming, but I have now moved on to think about something else, which gives yet another opportunity for coincidence. And so the opportunities for coincidence go on throughout the day and every day. But the negative occurrences, the failures to coincide, are not noticed and not reported.
Our propensity to see significance and pattern in coincidence, whether or not there is any real significance there, is part of a more general tendency to seek patterns. This tendency is laudable and useful. Many events and features in the world really are patterned in a non-random way and it is helpful to us, and to animals generally, to detect these patterns. The difficulty is to navigate between the Scylla of detecting apparent pattern when there isn't any, and the Charybdis of failing to detect pattern when there is. The science of statistics is quite largely concerned with steering this difficult course. But long before statistical methods were formalized, humans and indeed other animals were reasonably good intuitive statisticians. It is easy to make mistakes, however, in both directions.
Here are some true statistical patterns in nature which are not totally obvious, and which humans have not always known.
True pattern
Sexual intercourse is statistically followed by birth about 266 days later
Reason difficult to detect
The exact interval varies around the average of 266 days. Intercourse more often than not fails to result in conception. Intercourse is often frequent anyway, so it is not obvious that conception results from that rather than from, say, eating, which is also frequent.
True pattern
Conception is relatively probable in the middle of a woman's cycle, and relatively improbable near menstruation
Reason difficult to detect
See above. In addition, women who don't menstruate don't conceive. This is a spurious correlation which gets in the way and even, to a naive mind, suggests the opposite of the truth.
True pattern
Smoking causes lung cancer
Reason difficult to detect
Plenty of people who smoke don't get lung cancer. Many people get lung cancer who never smoked.
True pattern
In a time of bubonic plague, proximity to rats, and especially their fleas, tends to lead to infection
Reason difficult to detect
Lots of rats and fleas around anyway. Rats and fleas are associated with so many other things, such as dirt and 'bad air', that it is hard to know which of the many correlated factors is the important one. i. e. again, there are spurious correlations that get in the way.
Now here are some false patterns which humans have mistakenly thought they detected.
False pattern
Droughts can be brought to an end by a rain dance (or human sacrifice, or sprinkling goats' blood on a ferret's kidneys, or whatever arbitrary custom the particular theology lays down)
Reason easy to he misled
Occasionally, rains do chance to follow upon a rain dance (etc. ), and these rare lucky strikes lodge in the memory. When the rain dance, say, is not followed by rain, it is assumed that some detail went wrong with the ceremony, or that the gods are angry for some other reason: it is always easy enough to find a sufficiently plausible excuse.
False pattern
Comets and other astronomical events portend crises in human affairs
Reason easy to he misled
See above. Also, it is in the interests of astrologers to foster the myth, just as it is no doubt in the interests of priests and witch-doctors to foster the myths about rain dances and ferrets' kidneys.
False pattern
After a run of ill-luck, good luck becomes more likely
Reason easy to he misled
If bad luck persists, we assume that the run of bad luck hasn't ended yet, and we look forward all the more to its eventual end. If bad luck does not persist, the prophecy is seen as fulfilled. We subconsciously define a
'run' of bad luck in terms of its end. Therefore it obviously has to be followed by good luck.
We are not the only animals to seek statistical patterns of non- randomness in nature, and we are not the only animals to make mistakes of the kind that might be called superstitious. Both these facts are neatly demonstrated in the apparatus called the Skinner box, after the famous American psychologist B. F. Skinner. A Skinner box is a simple but versatile piece of equipment for studying the psychology of, usually, a rat or a pigeon. It is a box with a switch or switches let into one wall which the pigeon (say) can operate by pecking. There is also an electrically operated feeding (or other rewarding) apparatus. The two are connected in such a way that pecking by the pigeon has some influence on the feeding apparatus. In the simplest case, every time the pigeon pecks the key it gets food. Pigeons readily learn the task. So do rats and, in suitably enlarged and reinforced Skinner boxes, so do pigs.
We know that the causal link between key peck and food is provided by electrical apparatus, but the pigeon doesn't. As far as the pigeon is concerned, pecking a key might as well be a rain dance. Moreover, the link can be quite a weak, statistical one. The apparatus may be set up so that, far from every peck being rewarded, only one in 10 pecks is rewarded. This can mean literally every tenth peck. Or, with a different setting of the apparatus, it can mean that one in 10 pecks on average is rewarded, but on any particular occasion the exact number of pecks
required is determined at random. Or there may be a clock which determines that one tenth of the time, on average, a peck will yield reward, but it is impossible to tell which tenth of the time. Pigeons and rats learn to press keys even when, one might think, you'd need to be quite a good statistician to detect the cause-effect relationship. They can be worked up to a schedule in which only a very small proportion of pecks is rewarded. Interestingly, habits learned when pecks are only occasionally rewarded are more durable than habits learned when all pecks are rewarded: the pigeon is less swiftly discouraged when the rewarding mechanism is switched off altogether. This makes intuitive sense if you think about it.
Pigeons and rats, then, are quite good statisticians, able to pick up slight, statistical laws of patterning in their world. Presumably this ability
serves them in nature as well as in the Skinner box. Life out there is rich in pattern; the world is a big, complicated Skinner box. Actions by a wild animal are frequently followed by rewards or punishments or other important events. The relationship between cause and effect is frequently not absolute but statistical. If a curlew probes mud with its long, curved bill, there is a certain probability that it will strike a worm. The relationship between probe events and worm events is statistical but real. A whole school of research on animals has grown up around so-called Optimal Foraging Theory. Wild birds show quite sophisticated abilities to assess, statistically, the relative food-richness of different areas and they switch their time between the areas accordingly.
Back in the laboratory, Skinner founded a large school of research using Skinner boxes for all kinds of detailed purposes. Then, in 1948, he tried
a brilliant variant on the standard technique. He completely severed the causal link between behaviour and reward. He set up the apparatus to 'reward' the pigeon from time to time no matter what the bird did. Now
all that the birds actually needed to do was sit back and wait for the reward. But in fact this is not what they did. Instead, in six out of eight cases, they built up -exactly as though they were learning a rewarded habit - what Skinner called 'superstitious' behaviour. Precisely what this consisted of varied from pigeon to pigeon. One bird spun itself round like a top, two or three turns anticlockwise, between 'rewards'. Another bird repeatedly thrust its head towards one particular upper corner of the box. A third bird showed 'tossing' behaviour, as if lifting an invisible curtain with its head. Two birds independently developed the habit of rhythmic, side-to-side 'pendulum swinging' of the head and body. This last habit, incidentally, must have looked rather like the courtship dance of some birds of paradise. Skinner used the word superstition because the birds behaved as if they thought that their habitual movement had a causal influence on the reward mechanism, when actually it didn't. It was the pigeon equivalent of a rain dance.
A superstitious habit, once established, might persist for hours, long after the reward mechanism had been switched off. The habits did not, however, remain unchanged in form. They drifted, like the progressive improvisations of an organist. In one typical case the pigeon's superstitious habit began as a sharp movement of the head from the middle position towards the left. As time went by, the movement became more energetic. Eventually the whole body moved in the same direction and a step or two would be taken with the legs. After many hours of 'topographic drift', this leftward stepping movement became the predominant feature of the habit. The superstitious habits themselves may have been derived from the species' natural repertoire, but it is still fair to say that performing them in this context, and performing them repeatedly, is unnatural for pigeons.
Skinner's superstitious pigeons were behaving like statisticians, but statisticians who have got it wrong. They were alert to the possibility of links between events in their world, especially links between rewards that they wanted and actions that it was in their power to take. A habit, such as shoving the head up into the corner of the cage, began by chance. The bird just happened to do it at the moment before the reward mechanism was due to clunk into action. Understandably enough, the bird developed the tentative hypothesis that there was a link between the two events. So it shoved its head into the comer again. Sure enough, by the luck of Skinner's timing mechanism, the reward came again. If the bird had tried the experiment of not shoving its head into the corner, it would have found that the reward came anyway.
A well-known 'psychic' goes on television, a lucrative engagement fixed up over lunch by his publicity agent. Staring out of ten million screens with hypnotically smouldering eyes (nice job by Make-up and Lighting), our imaginary seer intones that he feels a strange, spiritual rapport, a vibrating resonance of cosmic energy, with certain members of his audience. They will be able to tell who they are because, even as he utters his mystic incantation, their watches will stop. After only a brief pause, a telephone on his table rings and an amplified voice in awed tones announces that its owner's watch stopped dead within seconds of the clairvoyant's words. The caller adds that she had a premonition that this was going to happen even before she looked down at her watch, for something in her hero's burning eyes seemed to speak directly to her soul. She felt the 'vibrations' of 'energy'. Even as she is speaking, a second telephone rings. Yet another watch has stopped.
A third caller's grandfather clock stopped - surely a weightier feat than stopping a little watch whose delicate hairspring would naturally be more susceptible to psychic forces than the massive pendulum of the grandfather! Another viewer's watch actually stopped a little before the celebrated mystic made his pronouncement - is this not an even more impressive feat of psychic control? Yet another watch has been more impatiently susceptible to occult forces. It had stopped a whole day before, at the very moment when its owner looked at the famous mystic's photograph in the newspaper. The studio audience gasps its appreciation. This, surely, is psychic power beyond all scepticism, for it happened a whole day early! 'There are more things in heaven and earth, Horatio . . . '
What we need is less gasping and more thinking. This chapter is about how to take the sting out of coincidence by quietly sitting down and calculating the likelihood that it would have happened anyway. In the course of this, we shall discover that to disarm apparently uncanny coincidences is more interesting than gasping over them anyway.
Sometimes the calculation is easy. In a previous book I gave away the number of the combination lock on my bicycle. I felt safe in doing so because obviously my books would never be read by the kind of person who would steal a bicycle. Unfortunately somebody did steal it, and I now have a new lock with a new number, 4167. I find this number easy to remember. 41 is imprinted in my memory as the arbitrary code used to identify my clothes and shoes at boarding school. 67 is the age at which I am due to retire. Obviously there is no interesting coincidence here: whatever the number had been, I'd have searched my life for a mnemonic recipe and I'd have found it. But mark the sequel. On the day of writing this, I received from my Oxford college a letter saying:
Each person authorized to use the photocopiers is issued with a personal code number which permits access. Your new number is 4167.
My first thought was that I'd undoubtedly lose this piece of paper (I quickly lost its equivalent last year) and I must immediately think of a formula to fix it in my memory. Something similar to the mnemonic by which I remember my bicycle combination, perhaps? So I looked again at the number on the letter and, to borrow a neat line from Fred Hoyle's science fiction novel The Black Cloud, the figures on the piece of paper seemed to swell to a gigantic size. I didn't need a new mnemonic. The number was identical. I rushed to tell my wife of the amazing coincidence, but on more sober reflection I shouldn't have bothered.
The odds of this happening by chance alone are easily calculated. The first digit could have been anything from 0 to 9. So there is a one in 10 chance of getting a 4 and matching the bicycle lock. For each of these ten possibilities, the second digit could have been anything from 0 to 9, so again there is a one in 10 chance of matching the bike lock's second dial. The odds of matching the first two digits is therefore one in 100 and, following the logic through the other two digits, the odds of matching all four digits of the bicycle lock is one in 10,000. It is this large number that is our protection against theft.
The coincidence is impressive. But what should we conclude? Has something mysterious and providential been going on? Have guardian angels been at work behind the scenes? Have lucky stars swum into Uranus? No. There is no reason to suspect anything more than simple accident. The number of people in the world is so large compared with 10,000 that somebody, at this very moment, is bound to be experiencing a coincidence at least as startling as mine. It just happens that today was my day to notice such a coincidence. It isn't even an added coincidence that it happened to me on this particular day, while I was writing this chapter. I had in fact written the first draft of the chapter some weeks ago. I reopened it today, after the coincidence occurred, in order to insert this anecdote. I shall surely reopen it many times to revise and polish, and I shall not remove the references to 'today': they were accurate when written. This is another way in which we habitually inflate the impressiveness of coincidence in order to make a good story.
We can do a similar calculation for the television guru whose psychic miasma seemed to stop people's watches, but we'll have to use estimates rather than exact figures. Any given watch has a certain low probability of stopping at any moment. I don't know what this probability is, but here's the kind of way in which we could come to an estimate. If we take just digital watches, their battery typically runs out within a year. Approximately, then, a digital watch stops once per year. Presumably clockwork watches stop more often because people forget to wind them
and presumably digital watches stop less often because people sometimes remember to renew the battery ahead of time. But both kinds of watches probably stop as often again because they develop faults of one kind or another. So, let our estimate be that any given watch is likely to stop about once a year. It doesn't matter too much how accurate our estimate is. The principle will remain.
If somebody's watch stopped three weeks after the spell was cast, even the most credulous would prefer to put it down to chance. We need to decide how large a delay would have been judged by the audience as sufficiently simultaneous with the psychic's announcement to impress. About five minutes is certainly safe, especially since he can keep talking to each caller for a few minutes before the next call ceases to seem roughly simultaneous. There are about 100,000 five-minute periods in a year. The probability that any given watch, say mine, will stop in a designated five-minute period is about 1 in 100,000. Low odds, but there are 10 million people watching the show. If only half of them are wearing watches, we could expect about 25 of those watches to stop in any given minute. If only a quarter of these ring in to the studio, that is 6 calls, more than enough to dumbfound a naive audience. Especially when you add in the calls from people whose watches stopped the day before, people whose watches didn't stop but whose grandfather clocks did, people who died of heart attacks and their bereaved relatives phoned in to say that their 'ticker' gave out, and so on. This kind of coincidence is celebrated in the delightfully sentimental old song, 'Grandfather's Clock:'
Ninety years without slumbering. Tick, lock, tick, tock.
His life seconds numbering. Tick, tock, tick, tock.
It stopped . . . short. . . never to go again When the old man died.
Richard Feynman, in a 1963 lecture published posthumously in 1998, tells the story of how his first wife died at 9. 22 in the evening and the clock in her room was later found to have stopped at exactly 9. 22. There are those who would revel in the apparent mystery of this coincidence and feel that Feynman has taken away something precious when he gives us a simple, rational explanation of the mystery. The clock was old and erratic and was in the habit of stopping if tilted out of the horizontal. Feynman himself frequently repaired it. When Mrs Feynman died, the
nurse's duty was to record the exact time of death. She moved over to the clock, but it was in dark shadow. In order to see it, she picked it up - and tilted its face towards the light . . . The clock stopped. Is Feynman really spoiling something beautiful when he tells us what is surely the true - and very simple - explanation? Not for my money. For me, he is affirming the elegance and beauty of an orderly universe in which clocks stop for reasons, not to titillate human sentimental fancy.
At this point, I want to invent a technical term, and I hope you'll forgive an acronym. PETWHAC stands for Population of Events That Would Have Appeared Coincidental. Population may seem an odd word, but it is the correct statistical term. I won't keep using capital letters because they stand so unattractively on the page. Somebody's watch stopping within ten seconds of the psychic's incantation obviously belongs within the petwhac, but so do many other events. Strictly speaking, the grandfather clock's stopping should not be included. The mystic did not claim that he could stop grandfather clocks. Yet when somebody's grandfather clock did stop, they immediately telephoned in because they were, if anything, even more impressed than they would have been if their watch had stopped. The odd misconception is fostered that the psychic is even more powerful since he didn't even bother to mention that he could stop grandfather clocks, too! Similarly, he said nothing about watches stopping the day before or grandfathers' tickers suffering cardiac arrests.
People feel that such unanticipated events belong in the petwhac. It looks to them as though occult forces must have been at work. But when you start to think like this, the petwhac becomes really quite large, and therein lies the catch. If your watch stopped exactly 24 hours earlier, you would not have to be unduly gullible to embrace this event within the petwhac. If somebody's watch stopped exactly seven minutes before the spell, this might impress some people because seven is an ancient mystic number. And the same presumably goes for seven hours, seven days . . . The larger the petwhac, the less we ought to be impressed by the coincidence when it comes. One of the devices of an effective trickster is to make people think exactly the opposite.
By the way, I deliberately chose a more impressive trick for my imaginary psychic than is actually done with watches on television. The more familiar feat is to start watches that have stopped. The television
audience is invited to get up and fetch, out of drawers or attics, watches that have broken down, and hold them while the psychic performs some incantation or does some hypnotic eye work. What is really going on is that the warmth of the hand melts oil that has coagulated and this starts the watch ticking, if only briefly. Even if this happens in only a small proportion of cases, this proportion, multiplied by the large audience, will generate a satisfactory number of dumbfounded telephone calls. Actually,
as Nicholas Humphrey explains in his admirable expose of supernaturalism Soul Searching (1995), it has been demonstrated that more than 50 per cent of broken watches start, at least momentarily, if they are held in the hand.
Here's another example of a coincidence, where it is clear how to calculate the odds. We shall use it to go on and see how odds are sensitive to changing the petwhac. I once had a girlfriend who had the same birth date (though not in the same year) as my previous girlfriend. She told a friend of hers who believed in astrology, and the friend triumphantly asked how I could possibly justify my scepticism in the face of such overwhelming evidence that I had unwittingly been brought together with two successive women on the basis of their 'stars'. Once again, let's just think it through quietly. It is easy to calculate the odds that two people, chosen entirely at random, will have the same birthday. There are 565 days in the year. Whatever the birthday of the first person, the chance that the second will have the same birthday is 1 in 365 (forgetting leap years). If we pair people off in any particular way, such as taking the successive women friends of any one man, the odds that they will share their birthday are 1 in 365. If we take ten million men (less than the population of Tokyo or Mexico City), this apparently uncanny coincidence will have happened to more than 27,000 of them!
Now let's think about the petwhac and see how the apparent coincidence becomes less impressive as it swells. There are many other ways in which we could pair people off and still end up noticing an apparent coincidence. Two successive girlfriends with the same surname, although unrelated, for instance. Two business partners with the same birthday would also come within the petwhac; or two people with the same birthday sitting next to one another on an aeroplane. Yet, in a well- loaded Boeing 747, the odds are actually better than 50 per cent that at least one pair of neighbours will share a birthday. We don't usually notice this because we don't look over each other's shoulders as we fill in those tedious immigration forms. But if we did, somebody on most flights would go away muttering darkly about occult forces.
The birthday coincidence is famously phrased in a more dramatic way. If you have a roomful of only 25 people, mathematicians can prove that the odds are just greater than 50 per cent that at least two of them will share the same birthday. Two readers of an earlier draft of the book asked me to justify this astonishing statement. It's easier to calculate the odds that there won't be any shared birthdays and subtract from one. Forget about leap years because they're more trouble than they're worth. Suppose I bet you that with 25 people in a room, at least two will share a birthday. You bet, for the sake of argument, that there'll be no shared birthdays. We're going to do the calculation by working up to 23 people gradually,
starting with just one person in the room, and adding people one at a time. If at any point a match is found, I've won the bet, we stop the game and don't bother to add any more people. If we get to 23 people and there's still no match so far, you win the bet.
When the room contains only the first person, whom we may as well call A, the chance of 'no match so far' is trivially 1 (365 out of 365 chances). Now add a second person, B. The chance of a match is now one in 365. So the odds of 'no match so far' when B has joined A in the room are 364/365. Now add a third person, C, There's a one in 365 chance that C matches A and a one in 365 chance that C matches B, therefore the chance that he matches neither A nor B is 363/365 (he can't match both, because we already know that A doesn't match B). To get the total odds
of 'no match so far' we have to take this 363/365 and multiply it by the odds against a match in the previous round (s), in this case by 364/365. The same reasoning applies when we add the fourth person, D. The total odds of 'no match so far' are now 364/365 X 363/365 X 362/365. And so on until all 23 people are in the room. Each new person adds a new term that we have to bring in to the running multiplication sum, to compute 'no match so far'.
If you multiply this out for 23 terms (you have to go on down to 343/365) the answer comes to about 0. 49. This is the chance that there will not be any shared birthdays in the room. So there's a slightly greater than even chance that at least one pair of individuals in a committee of 23 will share a birthday. Most people's intuition would encourage them to bet against such a coincidence. But they'd be wrong. It is this kind of intuitive error that in general bedevils our assessment of 'uncanny' coincidences.
Here's an actual coincidence where, although it is a little harder, we can make a stab at estimating the odds approximately. My wife once bought for her mother a beautiful antique watch with a pink face. When she got it home and peeled off the price label she was amazed to find, engraved on the back of the watch, her mother's own initials, M. A. B. Uncanny? Eerie? Spine-crawling? Arthur Koestler, the famous novelist, would have read much into it. So would C. G. Jung, the widely admired psychologist and inventor of the 'collective unconscious', who also believed that a bookcase or a knife might be induced by psychic forces to explode spontaneously with a loud report. My wife, who has more sense, merely thought the coincidence of initials remarkably convenient and sufficiently amusing to justify telling the story to me - and here I am now telling it to a wider audience.
So, what really are the odds against a coincidence of this magnitude? We can begin by calculating them in a naive way. There are 26 letters in the
alphabet. If your mother has three initials and you find a watch engraved with three letters at random, the odds that the two will coincide is 1/26 x 1/26 x 1/26, or one in 17,576. There are about 55 million people in Britain. If every one of them bought an antique engraved watch we'd expect more than 3,000 of them to gasp with amazement when they discovered that the watch already bore their mother's initials.
But the odds are actually better than this. Our naive calculation made the incorrect assumption that each letter has a probability of 1/26 of being somebody's initial. This is the average probability for the alphabet as a whole, but some letters, such as X and Z, have a smaller probability. Others, including M, A and B, are commoner: think how much more impressed we'd be if the coinciding initials had been X. Q. Z. We can improve our estimate of odds by sampling a telephone directory.
Sampling is a respectable way of estimating something that we cannot count directly. The London directory is a good place to sample because it is large and London happens to be where my wife bought the watch and where her mother lived. The London telephone directory contains about 85,060 column inches, or about 1. 34 column miles, of private citizens' names. Of these, about 8,110 column inches are devoted to the Bs. This means that about 9. 5 per cent of Londoners have a surname beginning with B - much more frequent than the figure for an average letter: 1/26, or 3. 8 per cent.
So, the probability that a randomly chosen Londoner would have a surname beginning with B is about 0. 095 (~ 9. 5 per cent). What about
the corresponding probabilities that the forenames will begin with M or A? It would take too long to count forename initials right through the telephone book, and there'd be no point since the telephone book is itself only a sample. The easiest thing to do is take a subsample where forename initials are conveniently arranged in alphabetical order. This is true of the listings within any one surname. I shall take the commonest surname in England - Smith - and look at what proportion of the Smiths are M. Smith and what proportion are A. Smith. It is a reasonable hope that this will be approximately representative of the probabilities of forename initials for Londoners generally. It turns out that there are rather more than 20 column yards of Smiths altogether. Of these, 0. 075 them (53. 6 column inches) are M. Smiths. The A. Smiths fill 75-4 column inches, representing 0. 102 of all the Smiths,
If you are a Londoner and you have three initials, therefore, the chances of your initials being M. A. B. in that order are approximately 0. 102 X 0. 073 0-095 or about 0. 0007. Since the population of Britain is 55 million, this should mean that about 38,000 of them have the initials M. A. B. , but only if everybody among those 55 million has three initials. Obviously not everybody does but, looking down the telephone directory
again, it seems that at least a majority do. If we make the conservative assumption that only half of British people have three initials, that still means that more than 19,000 British people have identical initials to my wife's mother. Any one of them could have bought that watch and gasped with astonishment at the coincidence. Our calculation has shown that there is no reason to gasp.
Indeed, when we think harder about the petwhac, we find that we have even less right to be impressed. M. A. B. were the initial letters of my wife's mother's maiden name. Her married initials of M. A. W. would have seemed just as impressive had they been found on the watch. Surnames beginning with W are nearly as common in the telephone book as those beginning with B. This consideration approximately doubles the petwhac, by doubling the number of people in the country who would have been deemed, by a coincidence hunter, to have 'the same initials' as my wife's mother.
Moreover, if somebody bought a watch and found it to be engraved not with her mother's initials but with her own, she might consider it an even greater coincidence and more worthy to be embraced within the (ever-growing) petwhac.
The late Arthur Koestler, as I have already mentioned, was a great enthusiast of coincidences. Among the stories that he recounts in The Roots of Coincidence (1972) are several originally collected by his hero, the Austrian biologist Paul Kammerer (famous for publishing a faked experiment purportedly demonstrating the 'inheritance of acquired characteristics' in the midwife toad). Here is a typical Kammerer story quoted by Koestler:
On September 18, 1916, my wife, while waiting for her turn in the consulting rooms of Prof. Dr J. v. H. , reads the magazine Die Kunst; she is impressed by some reproductions of pictures by a painter named Schwalbach, and makes a mental note to remember his name because she would like to see the originals. At that moment the door opens and the receptionist calls out to the patients: 'Is Frau Schwalbach here? She is wanted on the telephone. '
It probably isn't worth trying to estimate the odds against this coincidence, but we can at least write down some of the things that we'd need to know. 'At that moment the door opens' is a little vague. Did the door open one second after she made the mental note to look up Schwalbach's paintings or 20 minutes? How long could the interval have been, leaving her still impressed by the coincidence? The frequency of the name Schwalbach is obviously relevant: we'd be less impressed if it had been Schmidt or Strauss; more impressed if it had been Twistleton- Wykeham-Fiennes or Knatchbull-Huguesson. My local library doesn't
have the Vienna telephone book, but a quick look in another large Germanic telephone directory, the Berlin one, yields half a dozen Schwalbachs: the name is not particularly common, therefore, and it is understandable that the lady was impressed. But we need to think further about the size of the petwhac. Similar coincidences could have happened to people in other doctors' waiting rooms; and in dentists' waiting rooms, government offices and so on; and not just in Vienna but anywhere else. The quantity to keep bearing in mind is the number of opportunities for coincidence that would have been thought, if they had occurred, just as remarkable as the one that actually did occur.
Now let's take another kind of coincidence, where it is even harder to know how to start calculating odds. Consider the often-quoted experience of dreaming of an old acquaintance for the first time in years and then getting a letter from him, out of the blue, the next day. Or of learning that he died in the night. Or of learning that he didn't die in the night but his father did. Or that his father didn't die but won the football pools. See how the petwhac grows out of control when we relax our vigilance?
Often, these coincidence stories are gathered together from a large field. The correspondence columns of popular newspapers contain letters sent in by individual readers who would not have written but for the amazing coincidence that had happened to them. In order to decide whether we should be impressed, we need to know the circulation figure for the newspaper. If it is 4 million, it would be surprising if we did not read daily of some stunning coincidence, since a coincidence only has to happen to one of the 4 million in order for us to have a good chance of being told about it in the paper. It is hard to calculate the probability of a particular coincidence happening to one person, say a long-forgotten old friend dying during the night we happen to dream about him. But whatever this probability is, it is surely far greater than one in 4 million.
So, there really is no reason for us to be impressed when we read in the newspaper of a coincidence that has happened to one of the readers, or to somebody, somewhere in the world. This argument against being impressed is entirely valid. Nevertheless, there may be something lurking here that still bothers us. You may be happy to agree that, from the point of view of a reader of a mass-circulation newspaper, we have no right to be impressed at a coincidence that happens to another of the millions of readers of the same newspaper who bothers to write in. But it is much harder to shake the feeling of spine-chilled awe when the coincidence happens to you yourself. This is not just personal bias. One can make a serious case for it. The feeling occurs to almost everybody I meet; if you ask anybody at random, there is a good chance that they will have at least one pretty uncanny story of coincidence to relate. On the face of it,
this undermines the sceptic's point about newspaper stories having been culled from a millions-strong readership - a huge catchment of opportunity.
Actually it doesn't undermine it, for the following reason. Each one of us, though only a single person, none the less amounts to a very large population of opportunities for coincidence. Each ordinary day that you or I live through is an unbroken sequence of events, or incidents, any of which is potentially a coincidence. I am now looking at a picture on my wall of a deep-sea fish with a fascinatingly alien face. It is possible that, at this very moment, the telephone will ring and the caller will identify himself as a Mr Fish. I'm waiting . . .
The telephone didn't ring. My point is that, whatever you may be doing in any given minute of the day, there probably is some other event - a phone call, say - which, if it were to happen, would with hindsight be rated an eerie coincidence. There are so many minutes in every individual's lifetime that it would be quite surprising to find an individual who had never experienced a startling coincidence. During this particular minute, my thoughts have strayed to a schoolfellow called Haviland (I don't remember his Christian name, nor what he looked like) whom I haven't seen or thought of for 45 years. If, at this moment, an aeroplane manufactured by the de Haviland company were to fly past the window, I'd have a coincidence on my hands. In fact I have to report that no such plane has been forthcoming, but I have now moved on to think about something else, which gives yet another opportunity for coincidence. And so the opportunities for coincidence go on throughout the day and every day. But the negative occurrences, the failures to coincide, are not noticed and not reported.
Our propensity to see significance and pattern in coincidence, whether or not there is any real significance there, is part of a more general tendency to seek patterns. This tendency is laudable and useful. Many events and features in the world really are patterned in a non-random way and it is helpful to us, and to animals generally, to detect these patterns. The difficulty is to navigate between the Scylla of detecting apparent pattern when there isn't any, and the Charybdis of failing to detect pattern when there is. The science of statistics is quite largely concerned with steering this difficult course. But long before statistical methods were formalized, humans and indeed other animals were reasonably good intuitive statisticians. It is easy to make mistakes, however, in both directions.
Here are some true statistical patterns in nature which are not totally obvious, and which humans have not always known.
True pattern
Sexual intercourse is statistically followed by birth about 266 days later
Reason difficult to detect
The exact interval varies around the average of 266 days. Intercourse more often than not fails to result in conception. Intercourse is often frequent anyway, so it is not obvious that conception results from that rather than from, say, eating, which is also frequent.
True pattern
Conception is relatively probable in the middle of a woman's cycle, and relatively improbable near menstruation
Reason difficult to detect
See above. In addition, women who don't menstruate don't conceive. This is a spurious correlation which gets in the way and even, to a naive mind, suggests the opposite of the truth.
True pattern
Smoking causes lung cancer
Reason difficult to detect
Plenty of people who smoke don't get lung cancer. Many people get lung cancer who never smoked.
True pattern
In a time of bubonic plague, proximity to rats, and especially their fleas, tends to lead to infection
Reason difficult to detect
Lots of rats and fleas around anyway. Rats and fleas are associated with so many other things, such as dirt and 'bad air', that it is hard to know which of the many correlated factors is the important one. i. e. again, there are spurious correlations that get in the way.
Now here are some false patterns which humans have mistakenly thought they detected.
False pattern
Droughts can be brought to an end by a rain dance (or human sacrifice, or sprinkling goats' blood on a ferret's kidneys, or whatever arbitrary custom the particular theology lays down)
Reason easy to he misled
Occasionally, rains do chance to follow upon a rain dance (etc. ), and these rare lucky strikes lodge in the memory. When the rain dance, say, is not followed by rain, it is assumed that some detail went wrong with the ceremony, or that the gods are angry for some other reason: it is always easy enough to find a sufficiently plausible excuse.
False pattern
Comets and other astronomical events portend crises in human affairs
Reason easy to he misled
See above. Also, it is in the interests of astrologers to foster the myth, just as it is no doubt in the interests of priests and witch-doctors to foster the myths about rain dances and ferrets' kidneys.
False pattern
After a run of ill-luck, good luck becomes more likely
Reason easy to he misled
If bad luck persists, we assume that the run of bad luck hasn't ended yet, and we look forward all the more to its eventual end. If bad luck does not persist, the prophecy is seen as fulfilled. We subconsciously define a
'run' of bad luck in terms of its end. Therefore it obviously has to be followed by good luck.
We are not the only animals to seek statistical patterns of non- randomness in nature, and we are not the only animals to make mistakes of the kind that might be called superstitious. Both these facts are neatly demonstrated in the apparatus called the Skinner box, after the famous American psychologist B. F. Skinner. A Skinner box is a simple but versatile piece of equipment for studying the psychology of, usually, a rat or a pigeon. It is a box with a switch or switches let into one wall which the pigeon (say) can operate by pecking. There is also an electrically operated feeding (or other rewarding) apparatus. The two are connected in such a way that pecking by the pigeon has some influence on the feeding apparatus. In the simplest case, every time the pigeon pecks the key it gets food. Pigeons readily learn the task. So do rats and, in suitably enlarged and reinforced Skinner boxes, so do pigs.
We know that the causal link between key peck and food is provided by electrical apparatus, but the pigeon doesn't. As far as the pigeon is concerned, pecking a key might as well be a rain dance. Moreover, the link can be quite a weak, statistical one. The apparatus may be set up so that, far from every peck being rewarded, only one in 10 pecks is rewarded. This can mean literally every tenth peck. Or, with a different setting of the apparatus, it can mean that one in 10 pecks on average is rewarded, but on any particular occasion the exact number of pecks
required is determined at random. Or there may be a clock which determines that one tenth of the time, on average, a peck will yield reward, but it is impossible to tell which tenth of the time. Pigeons and rats learn to press keys even when, one might think, you'd need to be quite a good statistician to detect the cause-effect relationship. They can be worked up to a schedule in which only a very small proportion of pecks is rewarded. Interestingly, habits learned when pecks are only occasionally rewarded are more durable than habits learned when all pecks are rewarded: the pigeon is less swiftly discouraged when the rewarding mechanism is switched off altogether. This makes intuitive sense if you think about it.
Pigeons and rats, then, are quite good statisticians, able to pick up slight, statistical laws of patterning in their world. Presumably this ability
serves them in nature as well as in the Skinner box. Life out there is rich in pattern; the world is a big, complicated Skinner box. Actions by a wild animal are frequently followed by rewards or punishments or other important events. The relationship between cause and effect is frequently not absolute but statistical. If a curlew probes mud with its long, curved bill, there is a certain probability that it will strike a worm. The relationship between probe events and worm events is statistical but real. A whole school of research on animals has grown up around so-called Optimal Foraging Theory. Wild birds show quite sophisticated abilities to assess, statistically, the relative food-richness of different areas and they switch their time between the areas accordingly.
Back in the laboratory, Skinner founded a large school of research using Skinner boxes for all kinds of detailed purposes. Then, in 1948, he tried
a brilliant variant on the standard technique. He completely severed the causal link between behaviour and reward. He set up the apparatus to 'reward' the pigeon from time to time no matter what the bird did. Now
all that the birds actually needed to do was sit back and wait for the reward. But in fact this is not what they did. Instead, in six out of eight cases, they built up -exactly as though they were learning a rewarded habit - what Skinner called 'superstitious' behaviour. Precisely what this consisted of varied from pigeon to pigeon. One bird spun itself round like a top, two or three turns anticlockwise, between 'rewards'. Another bird repeatedly thrust its head towards one particular upper corner of the box. A third bird showed 'tossing' behaviour, as if lifting an invisible curtain with its head. Two birds independently developed the habit of rhythmic, side-to-side 'pendulum swinging' of the head and body. This last habit, incidentally, must have looked rather like the courtship dance of some birds of paradise. Skinner used the word superstition because the birds behaved as if they thought that their habitual movement had a causal influence on the reward mechanism, when actually it didn't. It was the pigeon equivalent of a rain dance.
A superstitious habit, once established, might persist for hours, long after the reward mechanism had been switched off. The habits did not, however, remain unchanged in form. They drifted, like the progressive improvisations of an organist. In one typical case the pigeon's superstitious habit began as a sharp movement of the head from the middle position towards the left. As time went by, the movement became more energetic. Eventually the whole body moved in the same direction and a step or two would be taken with the legs. After many hours of 'topographic drift', this leftward stepping movement became the predominant feature of the habit. The superstitious habits themselves may have been derived from the species' natural repertoire, but it is still fair to say that performing them in this context, and performing them repeatedly, is unnatural for pigeons.
Skinner's superstitious pigeons were behaving like statisticians, but statisticians who have got it wrong. They were alert to the possibility of links between events in their world, especially links between rewards that they wanted and actions that it was in their power to take. A habit, such as shoving the head up into the corner of the cage, began by chance. The bird just happened to do it at the moment before the reward mechanism was due to clunk into action. Understandably enough, the bird developed the tentative hypothesis that there was a link between the two events. So it shoved its head into the comer again. Sure enough, by the luck of Skinner's timing mechanism, the reward came again. If the bird had tried the experiment of not shoving its head into the corner, it would have found that the reward came anyway.
