No More Learning

? ? ? ? ? ? So Eurasia conquered the world not because Eurasians are smarter but because they could best take advantage of the principle that many heads are better than one. The "culture" of any of the conquering nations of Europe, such as Britain, is in fact a greatest-hits collection of inventions assembled across thousands of miles and years. The collection is made up of cereal crops and alphabetic writing from the Middle East, gunpowder and paper from China, domesticated horses from Ukraine, and many others. But the necessarily insular cultures of Australia, Africa, and the Americas had to make do with a few {69} homegrown technologies, and as a result they were no match for their pluralistic conquerors. Even within Eurasia and (later) the Americas, cultures that were isolated by mountainous geography -- for example, in the Appalachians, the Balkans, and the Scottish highlands -- remained backward for centuries in comparison with the vast network of people around them.
The extreme case, Diamond points out, is Tasmania. The Tasmanians, who were nearly exterminated by Europeans in the nineteenth century, were the most technologically primitive people in recorded history. Unlike the Aborigines on the Australian mainland, the Tasmanians had no way of making tire, no boomerangs or spear throwers, no specialized stone tools, no axes with handles, no canoes, no sewing needles, and no ability to fish. Amazingly, the archaeological record shows that their ancestors from the Australian mainland had arrived with these technologies ten thousand years before. But then the land bridge connecting Tasmania to the mainland was submerged and the island was cut off from the rest of the world. Diamond speculates that any technology can be lost from a culture at some point in its history. Perhaps a raw material came to be in short supply and people stopped making the products that depended on it. Perhaps all the skilled artisans in a generation were killed by a freak storm. Perhaps some prehistoric Luddite or ayatollah imposed a taboo on the practice for one inane reason or another. Whenever this happens n a culture that rubs up against other ones, the lost technology can eventually be reacquired as the people clamor for the higher standard of living enjoyed by their neighbors. But in lonely Tasmania, people would have had to reinvent the proverbial wheel every time it was lost, and so their standard of living ratcheted downward.
The ultimate irony of the Standard Social Science Model is that it failed to accomplish the very goal that brought it into being: explaining the different fortunes of human societies without invoking race. The best explanation today is thoroughly cultural, but it depends on seeing a culture as a product of human desires rather than as a shaper of them. History and culture, then, can be grounded in psychology, which can be grounded in computation, neuroscience, genetics, and evolution. But this kind of talk sets off alarms in the minds of many nonscientists. They fear that consilience is a smokescreen for a hostile takeover of the humanities, arts, and social sciences by philistines in white coats. The richness of their subject matter would be dumbed down into a generic palaver about neurons, genes, and evolutionary urges. This scenario is often called "reductionism," and I will conclude the chapter by showing why consilience does not call for it.
Reductionism, like cholesterol, comes in good and bad forms. Bad reductionism -- also called "greedy reductionism" or "destructive reductionism" -- consists of trying to explain a phenomenon in terms of its smallest or simplest
{70} constituents. Greedy reductionism is not a straw man. I know several scientists who believe (or at least say to granting agencies) that we will make break-throughs in education, conflict resolution, and other social concerns by studying the biophysics of neural membranes or the molecular structure of the synapse. But greedy reductionism is far from the majority view, and it is easy to show why it is wrong. As the philosopher Hilary Putnam has pointed out, even the simple fact that a square peg won't fit into a round hole cannot be explained in terms of molecules and atoms but only at a higher level of analysis involving rigidity (regardless of what makes the peg rigid) and geometry. 26 And if anyone really thought that sociology or literature or history could be replaced by biology, why stop there? Biology could in turn be ground up into chemistry, and chemistry into physics, leaving one struggling to explain the causes of World War I in terms of electrons and quarks. Even if World War I consisted of nothing but a very, very large number of quarks in a very, very complicated pattern of motion, no insight is gained by describing it that way.
Good reductionism (also called hierarchical reductionism) consists not of replacing one field of knowledge with another but of connecting or unifying them. The building blocks used by one field are put under a microscope by another. The black boxes get opened; the promissory notes get cashed. A geographer might explain why the coastline of Africa fits into the coastline of the Americas by saying that the landmasses were once adjacent but sat on different plates, which drifted apart. The question of why the plates move gets passed on to the geologists, who appeal to an upwelling of magma that pushes them apart. As for how the magma got so hot, they call in the physicists to explain the reactions in the Earth's core and mantle. None of the scientists is dispensable. An isolated geographer would have to invoke magic to move the continents, and an isolated physicist could not have predicted the shape of South America.
So, too, for the bridge between biology and culture. The big thinkers in the sciences of human nature have been adamant that mental life has to be understood at several levels of analysis, not just the lowest one. The linguist Noam Chomsky, the computational neuroscientist David Marr, and the ethologist Niko Tinbergen have independently marked out a set of levels of analysis for understanding a faculty of the mind. These levels include its function (what
? ? ? it accomplishes in an ultimate, evolutionary sense); its real-time operation (how it works proximately, from moment to moment); how it is implemented in neural tissue; how it develops in the individual; and how it evolved in the species. 27 For example, language is based on a combinatorial grammar designed to communicate an unlimited number of thoughts. It is utilized by people in real time via an interplay of memory lookup and rule application. It is implemented in a network of regions in the center of the left cerebral hemisphere that must coordinate memory, planning, word meaning, and grammar. {71} It develops in the first three years of life in a sequence from babbling to words to word combinations, including errors in which rules maybe overapplied. It evolved through modifications of a vocal tract and brain circuitry that had other uses in earlier primates, because the modifications allowed our ancestors to prosper in a socially interconnected, knowledge-rich lifestyle. None of these levels can be replaced by any of the others, but none can be fully understood in isolation from the others.
Chomsky distinguishes all of these from yet another level of analysis (one that he himself has little use for but that other language scholars invoke). The vantage points I just mentioned treat language as an internal, individual entity, such as the knowledge of Canadian English that I possess in my head. But Language can also be understood as an external entity: the "English language" as a whole, with its fifteen-hundred-year history, its countless dialects and hybrids spanning the globe, its half a million words in the Oxford English Dictionary. An external language is an abstraction that pools the internal languages of hundreds of millions of people living in different places and times. It could not exist without the internal languages in the minds of real humans conversing with one another, but it cannot be reduced to what any of them knows either. For example, the statement "English has a larger vocabulary than Japanese" could be true even if no English speaker has a larger vocabulary than any Japanese speaker.
The English language was shaped by broad historical events that did not take place inside a single head. They include the Scandinavian and Norman invasions in medieval times, which infected it with non-Anglo-Saxon words; the Great Vowel Shift of the fifteenth century, which scrambled the pronunciation of the long vowels and left its spelling system an irregular mess; the expansion of the British Empire, which budded off a variety of Englishes (American, Australian, Singaporean); and the development of global electronic media, which may rehomogenize the language as we all read the same web pages and watch the same television shows.
At the same time, none of these forces can be understood without taking into account the thought processes of flesh- and-blood people. They include the Britons who reanalyzed French words when they absorbed them into English, the children who failed to remember irregular past-tense forms like writhe-wrothe and crow-crew and converted them into regular verbs, the aristocrats who affected fussy pronunciations to differentiate themselves from the rabble, the mumblers who swallowed consonants to leave us made and had (originally maked and haved), and the clever speakers who first converted I had the house built to I had built the house and inadvertently gave English its perfect tense. Language is re-created every generation as it passes through the minds of the humans who speak it. 28
External language is, of course, a fine example of culture, the province of {72} social scientists and scholars in the humanities. The way that language can be understood at some half-dozen connected levels of analysis, from the brain and evolution to the cognitive processes of individuals to vast cultural systems, shows how culture and biology may be connected. The possibilities for connections in other spheres of human knowledge are plentiful, and we will encounter them throughout the book. The moral sense can illuminate legal and ethical codes. The psychology of kinship helps us understand sociopolitical arrangements. The mentality of aggression helps to make sense of war and conflict resolution. Sex differences are relevant to gender politics. Human aesthetics and emotion can enlighten our understanding of the arts.
What is the payoff for connecting the social and cultural levels of analysis to the psychological and biological ones? It is the thrill of discoveries that could never be made within the boundaries of a single discipline, such as universals of beauty, the logic of language, and the components of the moral sense. And it is the uniquely satisfying understanding we have enjoyed from the unification of the other sciences -- the explanation of muscles as tiny magnetic ratchets, of flowers as lures for insects, of the rainbow as a splaying of wavelengths that ordinarily blend into white. It is the difference between stamp collecting and detective work, between slinging around jargon and offering insight, between saying that something just is and explaining why it had to be that way as opposed to some other way it could have been. In a talk-show parody in Monty Python's Flying Circus, an expert on dinosaurs trumpets her new theory of the brontosaurus: "All brontosauruses are thin at one end; much, much thicker in the middle; and then thin again at the far end. " We laugh because she has not explained her subject in terms of deeper principles -- she has not "reduced" it, in the good sense. Even the word understand -- literally, "stand under" -- alludes to descending to a deeper level of analysis.
Our understanding of life has only been enriched by the discovery that living flesh is composed of molecular clockwork rather than quivering protoplasm, or that birds soar by exploiting the laws of physics rather than defying them. In the same way, our understanding of ourselves and our cultures can only be enriched by the discovery that our minds are composed of intricate neural circuits for thinking, feeling, and learning rather than blank slates,
? ? ? ? amorphous blobs, or inscrutable ghosts.
<< {73} >> Chapter 5
The Slate's Last Stand
Human nature is a scientific topic, and as new facts come in, our conception of it will change. Sometimes the facts may show that a theory grants our minds too much innate structure. For example, perhaps our language faculties are equipped not with nouns, verbs, adjectives, and prepositions but only with a distinction between more nounlike and more verblike parts of speech. At other times a theory may turn out to have granted our minds too little innate structure. No current theory of personality can explain why both members of a pair of identical twins reared apart liked to keep rubber bands around their wrists and pretend to sneeze in crowded elevators.
Also up for grabs is exactly how our minds use the information coming in from the senses. Once our faculties for language and social interaction are up and running, some kinds of learning may consist of simply recording information for future use, like the name of a person or the content of a new piece of legislation. Others may be more like setting a dial, flipping a switch, or computing an average, where the apparatus is in place but a parameter is left open so the mind can track variation in the local environment. Still others may use the information provided by all normal environments, such as the presence of gravity or the statistics of colors and lines in the visual field, to tune up our sensorimotor systems. There are yet other ways that nature and nurture might interact, and many will blur the distinction between the two.
This book is based on the estimation that whatever the exact picture turns out to be, a universal complex human nature will be part of it. I think we have reason to believe that the mind is equipped with a battery of emotions, drives, and faculties for reasoning and communicating, and that they have a common logic across cultures, are difficult to erase or redesign from scratch, were shaped by natural selection acting over the course of human evolution, and owe some of their basic design (and some of their variation) to information in the genome. This general picture is meant to embrace a variety of theories, present and future, and a range of foreseeable scientific discoveries. {74}
But the picture does not embrace just any theory or discovery. Conceivably scientists might discover that there is insufficient information in the genome to specify any innate circuitry, or no known mechanism by which it could be wired into the brain. Or perhaps they will discover that brains are made out of general-purpose stuff that can soak up just about any pattern in the sensory input and organize itself to accomplish just about any goal. The former discovery would make innate organization impossible; the latter would make it unnecessary. Those discoveries would call into question the very concept of human nature. Unlike the moral and political objections to the concept of human nature (objections that I discuss in the rest of this book), these would be scientific objections. If such discoveries are on the horizon, I had better look at them carefully.
This chapter is about three scientific developments that are sometimes interpreted as undermining the possibility of a complex human nature. The first comes from the Human Genome Project. When the sequence of the human genome was published in 2001, geneticists were surprised that the number of genes was lower than they had predicted. The estimates hovered around 34,000 genes, which lies well outside the earlier range of 50,000 to 100,000. 1 Some editorialists concluded that the smaller gene count refuted any claim about innate talents or tendencies, because the slate is too small to contain much writing. Some even saw it as vindicating the concept of free will: the smaller the machine, the more room for a ghost.
The second challenge comes from the use of computer models of neural networks to explain cognitive processes. These artificial neural networks can often be quite good at learning statistical patterns in their input. Some modelers from the school of cognitive science called connectionism suggest that generic neural networks can account for all of human cognition, with little or no innate tailoring for particular faculties such as social reasoning or language. In Chapter 2 we met the founders of connectionism, David Rumelhart and James McClelland, who suggested that people are smarter than rats only because they have more associative cortex and because their environment contains a culture to organize it.
The third comes from the study of neural plasticity, which examines how the brain develops in the womb and early childhood and how it records experience as the animal learns. Neuroscientists have recently shown how the brain changes in response to learning, practice, and input from the senses. One spin on these discoveries may be called extreme plasticity. According to this slant, the cerebral cortex -- the convoluted gray matter responsible for perception, thinking, language, and memory -- is a protean substance that can be shaped almost limitlessly by the structure and demands of the environment. The blank slate becomes the plastic slate.
? ? ? ? ? ? ? ? Connectionism and extreme plasticity are popular among cognitive {75} scientists at the West Pole, who reject a completely blank slate but want to restrict innate organization to simple biases in attention and memory. Extreme plasticity also appeals to neuroscientists who wish to boost the importance of their field for education and social policy, and to entrepreneurs selling products to speed up infant development, cure learning disabilities, or slow down aging. Outside the sciences, all three developments have been welcomed by some scholars in the humanities who want to beat back the encroachments of biology. 2 The lean genome, connectionism, and extreme plasticity are the Blank Slate's last stand.
The point of this chapter is that these claims are not vindications of the doctrine of the Blank Slate but products of the Blank Slate. Many people (including a few scientists) have selectively read the evidence, sometimes in bizarre ways, to fit with a prior belief that the mind cannot possibly have any innate structure, or with simplistic notions of how innate structure, if it did exist, would be encoded in the genes and develop in the brain.
I should say at the outset that I find these latest-and-best blank-slate theories highly implausible -- indeed, barely coherent. Nothing comes out of nothing, and the complexity of the brain has to come from somewhere. It cannot come from the environment alone, because the whole point of having a brain is to accomplish certain goals, and the environment has no idea what those goals are. A given environment can accommodate organisms that build dams, migrate by the stars, trill and twitter to impress the females, scent-mark trees, write sonnets, and so on. To one species, a snatch of human speech is a warning to flee; to another, it is an interesting new sound to incorporate into its own vocal repertoire; to a third, it is grist for grammatical analysis. Information in the world doesn't tell you what to do with it.
Also, brain tissue is not some genie that can grant its owner any power that would come in handy. It is a physical mechanism, an arrangement of matter that converts inputs to outputs in particular ways. The idea that a single generic substance can see in depth, control the hands, attract a mate, bring up children, elude predators, outsmart prey, and so on, without some degree of specialization, is not credible. Saying that the brain solves these problems because of its "plasticity" is not much better than saying it solves them by magic. Still, in this chapter I will examine the latest scientific objections to human nature carefully. Each of the discoveries is important on its own terms, even if it does not support the extravagant conclusions that have been drawn. And once the last supports for the Blank Slate have been evaluated, I can properly sum up the scientific case for the alternative.
The human genome is often seen as the essence of our species, so it is not surprising that when its sequence was announced in 2001 commentators rushed to give it the correct interpretation for human affairs. Craig Venter, {76} whose company had competed with a public consortium in the race to sequence the genome, said at a press conference that the smaller-than-expected gene count shows that "we simply do not have enough genes for this idea of biological determinism to be right. The wonderful diversity of the human species is not hard-wired in our genetic code. Our environments are critical. " In the United Kingdom, The Guardian headlined its story, "Revealed: The Secret of Human Behaviour. Environment, Not Genes, Key to Our Acts. "3 An editorial in another British newspaper concluded that "we are more free, it seems, than we had realized. " Moreover, the finding "offers comfort for the left, with its belief in the potential of all, however deprived their background. But it is damning for the right, with its fondness for ruling classes and original sin. "4
All this from the number 34,000! Which leads to the question, What number of genes would have proven that the diversity of our species was wired into our genetic code, or that we are less free than we had realized, or that the political right is right and the left is wrong? 50,000? 150,000? Conversely, if it turned out that we had only 20,000 genes, would that have made us even freer, or the environment even more important, or the political left even more comfortable? The fact is that no one knows what these numbers mean. No one has the slightest idea how many genes it would take to build a system of hard-wired modules, or a general-purpose learning program, or anything in between -- to say nothing of original sin or the superiority of the ruling class. In our current state of ignorance of how the genes build a brain, the number of genes in the human genome is just a number.
If you don't believe this, consider the roundworm Caenorhabditis elegans, which has about 18,000 genes. By the logic of the genome editorialists, it should be twice as free, be twice as diverse, and have twice as much potential as a human being. In fact, it is a microscopic worm composed of 959 cells grown by a rigid genetic program, with a nervous system consisting of exactly 302 neurons in a fixed wiring diagram. As far as behavior is concerned, it eats, mates, approaches and avoids certain smells, and that's about it. This alone should make it obvious that our freedom and diversity of behavior come from having a complex biological makeup, not a simple one.
Now, it is a genuine puzzle why humans, with their hundred trillion cells and hundred billion neurons, need only twice as many genes as a humble little worm. Many biologists believe that the human genes have been undercounted. The number of genes in a genome can only be estimated; right now they cannot literally be totted up. Gene- estimating programs look for sequences in the DNA that are similar to known genes and that are active enough to be caught in the act of building a protein. 5 Genes that are unique to humans or active only in the developing brain of the
? ? ? ? ? ? fetus -- the genes most relevant to human nature -- and other inconspicuous genes could evade the software and get left {77} out of the estimates. Alternative estimates of 57,000,75,000, and even 120,000 human genes are currently being bruited about. 6 Still, even if humans had six times as many genes as a roundworm rather than just twice as many, the puzzle would remain.
Most biologists who are pondering the puzzle don't conclude that humans are less complex than we thought. Instead they conclude that the number of genes in a genome has little to do with the complexity of the organism. 7 A single
gene does not correspond to a single component in such a way that an organism with 20,000 genes has 20,000 components, an organism with 30,000 genes has 30,000 components, and so on. Genes specify proteins, and some of the proteins do become the meat and juices of an organism. But other proteins turn genes on or off, speed up or slow down their activity, or cut and splice other proteins into new combinations. James Watson points out that we should recalibrate our intuitions about what a given number of genes can do: "Imagine watching a play with thirty thousand actors. You'd get pretty conrused. "
Depending on how the genes interact, the assembly process can be much more intricate for one organism than for another with the same number of genes. In a simple organism, many of the genes simply build a protein and dump it into the stew. In a complex organism, one gene may turn on a second one, which speeds up the activity of a third one (but only if a fourth one is active), which then turns off the original gene (but only if a fifth one is inactive), and so on. This defines a kind of recipe that can build a more complex organism out of the same number of genes. The complexity of an organism thus depends not just on its gene count but on the intricacy of the box-and-arrow diagram that captures how each gene impinges on the activity of the other genes.
8 And because adding a gene doesn't just add an ingredient but can multiply the number of ways that the genes can interact with one another, the complexity of organisms depends on the number of possible combinations of active and inactive genes in their genomes. The geneticist Jean-Michel Claverie suggests that it might be estimated by the number two (active versus inactive) raised to the power of the number of genes. By that measure, a human genome is not twice as complex as a roundworm genome but 216,000 (a one followed by 4. 800 zeroes) times as complex. 9
There are two other reasons why the complexity of the genome is not reflected in the number of genes it contains. One is that a given gene can produce not just one protein but several. A gene is typically broken into stretches of DNA that code for fragments of protein (exons) separated by stretches of DNA that don't (introns), a bit like a magazine article interrupted by ads. The segments of a gene can then be spliced together in multiple ways. A gene composed of exons A, B, C, and D might give rise to proteins corresponding to {78} ABC, ABD, ACD, and so on -- as many as ten different proteins per gene. This happens to a greater degree in complex organisms than in simple ones. 10
Second, the 34,000 genes take up only about 3 percent of the human genome. The rest consists of DNA that does not code for protein and that used to be dismissed as "junk. " But as one biologist recently put it, "The term 'junk DNA' is a reflection of our ignorance. "11 The size, placement, and content of the noncoding DNA can have dramatic effects on the way that nearby genes are activated to make proteins. Information in the billions of bases in the non-coding regions of the genome is part of the specification of a human being, above and beyond the information contained in the 34,000 genes.
The human genome, then, is fully capable of building a complex brain, in spite of the bizarre proclamations of how wonderful it is that people are almost as simple as worms. Of course "the wonderful diversity of the human species is not hard-wired in our genetic code," but we didn't need to count genes to figure that out -- we already know it from the fact that a child growing up in Japan speaks Japanese but the same child growing up in England would speak English. It is an example of a syndrome we will meet elsewhere in this book: scientific findings spin-doctored beyond recognition to make a moral point that could have been made more easily on other grounds.
The second scientific defense of the Blank Slate comes from connectionism, the theory that the brain is like the artificial neural networks simulated on computers to learn statistical patterns. 12
Cognitive scientists agree that the elementary processes that make up the instruction set of the brain -- storing and retrieving an association, sequencing elements, focusing attention -- are implemented in the brain as networks of densely interconnected neurons (brain cells). The question is whether a generic kind of network, after being shaped by the environment, can explain all of human psychology, or whether the genome tailors different networks to the demands of particular domains: language, vision, morality, fear, lust, intuitive psychology, and so on. The connectionists, of course, do not believe in a blank slate, but they do believe in the closest mechanistic equivalent, a general-purpose learning device.
What is a neural network? Connectionists use the term to refer not to real neural circuitry in the brain but to a kind of computer program based on the metaphor of neurons and neural circuits. In the most common approach, a "neuron" carries information by being more or less active. The activity level indicates the presence or absence (or intensity or degree of confidence) of a simple feature of the world. The feature may be a color, a line with a certain slant, a letter
? ? ? ? ? ? ? ? ? of the alphabet, or a property of an animal such as having four legs.
A network of neurons can represent different concepts, depending on {79} which ones are active. If neurons for "yellow," "flies," and "sings" are active, the network is thinking about a canary; if neurons for "silver," "flies," and "roars" are active, it is thinking about an airplane. An artificial neural network computes in the following manner. Neurons are linked to other neurons by connections that work something like synapses. Each neuron counts up the inputs from other neurons and changes its activity level in response. The network learns by allowing the input to change the strengths of the connections. The strength of a connection determines the likelihood that the input neuron will excite or inhibit the output neuron.
Depending on what the neurons stand for, how they are innately wired, and how the connections change with training, a connectionist network can learn to compute various things. If everything is connected to everything else, a network can soak up the correlations among features in a set of objects. For example, after exposure to descriptions of many birds it can predict that feathered singing things tend to fly or that feathered flying things tend to sing or that singing flying things tend to have feathers. If a network has an input layer connected to an output layer, it can learn associations between ideas, such as that small soft flying things are animals but large metallic flying things are vehicles. If its output layer feeds back to earlier layers, it can crank out ordered sequences, such as the sounds making up a word.
The appeal of neural networks is that they automatically generalize their training to similar new items. If a network has been trained that tigers eat Frosted Flakes, it will tend to generalize that lions eat Frosted Flakes, because eating Frosted Flakes" has been associated not with "tigers" but with simpler features like "roars" and "has whiskers," which make up part of the representation of h'ons, too. The school of connectionism, like the school of associationism championed by Locke, Hume, and Mill, asserts that these generalizations are the crux of intelligence. If so, highly trained but otherwise generic neural networks can explain intelligence.
Computer modelers often set their models on simplified toy problems to ? rove that they can work in principle. The question then becomes whether the models can "scale up" to more realistic problems, or whether, as skeptics say, the modeler "is climbing trees to get to the moon. " Here we have the problem with connectionism. Simple connectionist networks can manage impressive displays of memory and generalization in circumscribed problems like reading a list of words or learning stereotypes of animals. But they are simply too underpowered to duplicate more realistic feats of human intelligence like understanding a sentence or reasoning about living things.
Humans don't just loosely associate things that resemble each other, or things that tend to occur together. They have combinatorial minds that enterrain propositions about what is true of what, and about who did what to {80} whom, when and where and why. And that requires a computational architecture that is more sophisticated than the uniform tangle of neurons used in generic connectionist networks. It requires an architecture equipped with logical apparatus like rules, variables, propositions, goal states, and different kinds of data structures, organized into larger systems. Many cognitive scientists have made this point, including Gary Marcus, Marvin Minsky, Seymour Papert, Jerry Fodor, Zenon Pylyshyn, John Anderson, Tom Bever, and Robert Hadley, and it is acknowledged as well by neural network modelers who are not in the connectionist school, such as John Hummel, Lokendra Shastri, and Paul Smolensky. 13 I have written at length on the limits of connectionism, both in scholarly papers and in popular books; here is a summary of my own case. 14
In a section called "Connectoplasm" in How the Mind Works, I laid out some simple logical relationships that underlie our understanding of a complete thought (such as the meaning of a sentence) but that are difficult to represent in generic networks. 15 One is the distinction between a kind and an individual: between ducks in general and this duck in particular. Both have the same features (swims, quacks, has feathers, and so on), and both are thus represented by the same set of active units in a standard connectionist model. But people know the difference.
A second talent is compositionality: the ability to entertain a new, complex thought that is not just the sum of the simple thoughts composing it but depends on their relationships. The thought that cats chase mice, for example, cannot be captured by activating a unit each for "cats," "mice," and "chase," because that pattern could just as easily stand for mice chasing cats.
A third logical talent is quantification (or the binding of variables): the difference between fooling some of the people all of the time and fooling all of the people some of the time. Without the computational equivalent of x's,y's, parentheses, and statements like "For all x," a model cannot tell the difference.
A fourth is recursion: the ability to embed one thought inside another, so that we can entertain not only the thought that Elvis lives, but the thought that the National Enquirer reported that Elvis lives, that some people believe the National Enquirer report that Elvis lives, that it is amazing that some people believe the National Enquirer report that Elvis lives, and so on. Connectionist networks would superimpose these propositions and thereby confuse their various subjects and predicates.
A final elusive talent is our ability to engage in categorical, as opposed to fuzzy, reasoning: to understand that Bob
? ? ? ? ? Dylan is a grandfather, even though he is not very grandfatherly, or that shrews are not rodents, though they look just like mice. With nothing but a soup of neurons to stand for an object's properties, and no provision for rules, variables, and definitions, the networks fall back on stereotypes and are bamboozled by atypical examples. {81}
In Words and Rules I aimed a microscope on a single phenomenon of language that has served as a test case for the ability of generic associative networks to account for the essence of language: assembling words, or pieces of words, into new combinations. People don't just memorize snatches of language but create new ones. A simple example is the English past tense. Given a neologism like to spam or to snarf, people don't have to run to the dictionary to look up their past-tense forms; they instinctively know that they are spammed and snarfed. The talent for assembling new combinations appears as early as age two, when children overapply the past-tense suffix to irregular verbs, as in We holded the baby rabbits and Horton heared a Who. 16
The obvious way to explain this talent is to appeal to two kinds of computational operations in the mind. Irregular forms like held and heard are stored in and retrieved from memory, just like any other word. Regular forms like walk- walked can be generated by a mental version of the grammatical rule "Add -ed to the verb. " The rule can apply whenever memory fails. It may be used when a word is unfamiliar and no past-tense form had been stored in memory, as in to spam, and it may be used by children when they cannot recall an irregular form like heard and need some way of marking its tense. Combining a suffix with a verb is a small example of an important human talent: combining words and phrases to create new sentences and thereby express new thoughts. It is one of the new ideas of the cognitive revolution introduced in Chapter 3, and one of the logical challenges for connectionism I listed in the preceding discussion.
Connectionists have used the past tense as a proving ground to see if they could duplicate this textbook example of human creativity without using a rule and without dividing the labor between a system for memory and a system for grammatical combination. A series of computer models have tried to generate past-tense forms using simple pattern associator networks. The networks typically connect the sounds in verbs with the sounds in the past-tense form: -am with -ammed, -ing with -ung, and so on. The models can then generate new forms by analogy, just like the generalization from tigers to lions: trained on crammed, a model can guess spammed; trained on folded, it tends to say holded.
But human speakers do far more than associate sounds with sounds, and the models thus fail to do them justice. The failures come from the absence of machinery to handle logical relationships. Most of the models are baffled by new words that sound different from familiar words and hence cannot be generalized by analogy. Given the novel verb to frilg, for example, they come up not with frilged, as people do, but with an odd mishmash like freezled. That is because they lack the device of a variable, like x in algebra or "verb" in grammar, which can apply to any member of a category, regardless of how familiar {82} its properties are. (This is the gadget that allows people to engage in categorical rather than fuzzy reasoning. ) The networks can only associate bits of sound with bits of sound, so when confronted with a new verb that does not sound like anything they were trained on, they assemble a pastiche of the most similar sounds they can find in their network.
The models also cannot properly distinguish among verbs that have the same sounds but different past-tense forms, such as ring the bell-rang the bell and ring the city-ringed the city. That is because the standard models represent
only sound and are blind to the grammatical differences among verbs that call for different conjugations. The key difference here is between simple roots like ring in the sense of "resonate" (past tense rang) and complex verbs derived from nouns like ring in the sense of "form a ring around" (past tense ringed). To register that difference, a language-using system has to be equipped with compositional data structures (such as "a verb made from the noun ring") and not just a beanbag of units.
Yet another problem is that connectionist networks track the statistics of the input closely: how many verbs of each sound pattern they have encountered. That leaves them unable to account for the epiphany in which young children discover the -ed rule and start making errors like holded and heared. Connectionist modelers can induce these errors only by bombarding the network with regular verbs (so as to burn in the -ed) in a way that is unlike anything real children experience. Finally, a mass of evidence from cognitive neuroscience shows that grammatical combination (including regular verbs) and lexical lookup (including irregular verbs) are handled by different systems in the brain rather than by a single associative network.
It's not that neural networks are incapable of handling the meanings of sentences or the task of grammatical conjugation. (They had better not be, since the very idea that thinking is a form of neural computation requires that some kind of neural network duplicate whatever the mind can do. ) The problem lies in the credo that one can do everything with a generic model as long as it is sufficiently trained. Many modelers have beefed up, retrofitted, or combined networks into more complicated and powerful systems. They have dedicated hunks of neural hardware to abstract symbols like "verb phrase" and "proposition" and have implemented additional mechanisms (such as synchronized firing patterns) to bind them together in the equivalent of compositional, recursive symbol structures.
? ? ? ? They have installed banks of neurons for words, or for English suffixes, or for key grammatical distinctions. They have built hybrid systems, with one network that retrieves irregular forms from memory and another that combines a verb with a suffix. 17
A system assembled out of beefed-up subnetworks could escape all the criticisms. But then we would no longer be talking about a generic neural network! We would be talking about a complex system innately tailored to {83} compute a task that people are good at. In the children's story called "Stone Soup," a hobo borrows the use of a woman's kitchen ostensibly to make soup from a stone. But he gradually asks for more and more ingredients to balance the flavor until he has prepared a rich and hearty stew at her expense. Connectionist modelers who claim to build intelligence out of generic neural networks without requiring anything innate are engaged in a similar business. The design choices that make a neural network system smart -- what each of the neurons represents, how they are wired together, what kinds of networks are assembled into a bigger system, in which way -- embody the innate organization of the part of the mind being modeled. They are typically hand-picked by the modeler, like an inventor rummaging through a box of transistors and diodes, but in a real brain they would have evolved by natural selection (indeed, in some networks, the architecture of the model does evolve by a simulation of natural selection). 18 The only alternative is that some previous episode of learning left the networks in a state ready for the current learning, but of course the buck has to stop at some innate specification of the first networks that kick off the learning process.
So the rumor that neural networks can replace mental structure with statistical learning is not true. Simple, generic networks are not up to the demands of ordinary human thinking and speaking; complex, specialized networks are a stone soup in which much of the interesting work has been done in setting up the innate wiring of the network. Once this is recognized, neural network modeling becomes an indispensable complement to the theory of a complex human nature rather than a replacement for it. 19 It bridges the gap between the elementary steps of cognition and the physiological activity of the brain and thus serves as an important link in the long chain of explanation between biology and culture. ~
FOR MOST OF its history, neuroscience was faced with an embarrassment: the brain looked as if it were innately specified in every detail. When it comes to the body, we can see many of the effects of a person's life experience: it may be tanned or pale, callused or soft, scrawny or plump or chiseled. But no such marks could be found in the brain. Now, something has to be wrong with this picture. People learn, and learn massively: they learn their language, their culture, their know-how, their database of facts. Also, the hundred trillion connections in the brain cannot possibly be specified individually by a 750-megabyte genome. The brain somehow must change in response to its input; the only question is how.
We are finally beginning to understand how. The study of neural plasticity is hot. Almost every week sees a discovery about how the brain gets wired in the womb and tuned outside it. After all those decades in which no one could find anything that changed in the brain, it is not surprising that the {84} discovery of plasticity has given the nature-nurture pendulum a push. Some people describe plasticity as a harbinger of an expansion of human potential in which the powers of the brain will be harnessed to revolutionize childrearing, education, therapy, and aging. And several manifestos have proclaimed that plasticity proves that the brain cannot have any significant innate organization. 20 In Rethinking Innateness, Jeffrey Elman and a team of West Pole connectionists write that predispositions to think about different things in different ways (language, people, objects, and so on) may be implemented in the brain only as "attention-grabbers" that ensure that the organism will receive "massive experience of certain inputs prior to subsequent learning. "21 In a "constructivist manifesto," the theoretical neuroscientists Stephen Quartz and Terrence Sejnowski write that "although the cortex is not a tabula rasa . . . it is largely equipotential at early stages," and therefore that innatist theories "appear implausible. "22
Neural development and plasticity unquestionably make up one of the great frontiers of human knowledge. How a linear string of DNA can direct the assembly of an intricate three-dimensional organ that lets us think, feel, and learn is a problem to stagger the imagination, to keep neuroscientists engaged for decades, and to belie any suggestion that we are approaching "the end of science. "
And the discoveries themselves are fascinating and provocative. The cerebral cortex (outer gray matter) of the brain has long been known to be divided into areas with different functions. Some represent particular body parts; others represent the visual field or the world of sound; still others concentrate on aspects of language or thinking. We now know that with learning and practice some of their boundaries can move around. (This does not mean that the brain tissue literally grows or shrinks, only that if the cortex is probed with electrodes or monitored with a scanner, the boundary where one ability leaves off and the next one begins can shift. ) Violinists, for example, have an expanded region of cortex representing the fingers of the left hand. 23 If a person or a monkey is trained on a simple task like recognizing shapes or attending to a location in space, neuroscientists can watch as parts of the cortex, or even
? ? ? ? ? ? ? ? ? individual neurons, take on the job. 24
The reallocation of brain tissue to new tasks is especially dramatic when people lose the use of a sense or body part. Congenitally blind people use their visual cortex to read Braille. 25 Congenitally deaf people use part of their auditory cortex to process sign language. 26 Amputees use the part of the cortex formerly serving the missing limb to represent other parts of their bodies. 27 Young children can grow up relatively normal after traumas to the brain that would turn adults into basket cases -- even removal of the entire left hemisphere, which in adults underlies language and logical reasoning. 28 All this {85} suggests that the allocation of brain tissue to perceptual and cognitive processes is not done permanently and on the basis of the exact location of the tissue in the skull, but depends on how the brain itself processes information.
This dynamic allocation of tissue can also be seen as the brain puts itself together in the womb. Unlike a computer that gets assembled in a factory and is turned on for the first time when complete, the brain is active while it is being assembled, and that activity may take part in the assembly process. Experiments on cats and other mammals have shown that if a brain is chemically silenced during fetal development it may end up with significant abnormalities. 29 And patches of cortex develop differently depending on the kind of input they receive. In an experimental tour de force, the neuroscientist Mriganka Sur literally rewired the brains of ferrets so that signals from their eyes fed into the primary auditory cortex, the part of the brain that ordinarily receives signals from the ears. 30 When he then probed the auditory cortex with electrodes, he found that it acted in many ways like the visual cortex. Locations in the visual field were laid out like a map, and individual neurons responded to lines and stripes at a particular orientation and direction of movement, similar to the neurons in an ordinary visual cortex. The ferrets could even use their rewired brains to move toward objects that were detectable by sight alone. The input to the sensory cortex must help to organize it: visual input makes the auditory cortex work something like the visual cortex.
What do these discoveries mean? Do they show that the brain is "able to be shaped, molded, modeled, or sculpted," as the dictionary definition of plastic would suggest? In the rest of this chapter I will show you that the answer is no. 31 Discoveries of how the brain changes with experience do not show that learning is more powerful than we thought, that the brain can be dramatically reshaped by its input, or that the genes do not shape the brain. Indeed, demonstrations of the plasticity of the brain are less radical than they first appear: the supposedly plastic regions of cortex are doing pretty much the same thing they would have been doing if they had never been altered. And the most recent discoveries on brain development have refuted the idea that the brain is largely plastic. Let me go over these points in turn.
The fact that the brain changes when we learn is not, as some have claimed, a radical discovery with profound implications for nature and nurture or human potential. Dmitri Karamazov could have deduced it in his nineteenth- century prison cell as he mulled over the fact that thinking comes from quivering nerve tails rather than an immaterial soul. If thought and action are products of the physical activity of the brain, and if thought and action can be affected by experience, then experience has to leave a trace in the physical structure of the brain. {86}
So there is. no scientific question as to whether experience, learning, and practice affect the brain; they surely do if we are even vaguely on the right track. It is not surprising that people who can play the violin have different brains from those who cannot, or that masters of sign language or of Braille have different brains from people who speak and read. Your brain changes when you are introduced to a new person, when you hear a bit of gossip, when you watch the Oscars, when you polish your golf stroke -- in short, whenever an experience leaves a trace in the mind.