No More Learning

Boas helped overthrow the bad racial science of the nineteenth century that attributed these disparities to differences in how far each race had biologically evolved. In its place his successors stipulated that behavior is determined by culture and that culture is autonomous from biology. 22 Unfortunately, that left the dramatic differences among cultures unexplained, as if they were random outcomes of the lottery in Babylon. Indeed, the differences were not just unexplained but unmentionable, out of a fear that people would misinterpret the observation that some cultures were more technologically sophisticated than others as some kind of moral judgment that advanced societies were better than primitive ones. But no one can fail to notice that some cultures can accomplish things that all people want (like health and comfort) better than others. The dogma that cultures vary capriciously is a feeble refutation of any private opinion that some races have what it takes to develop science, technology, and government and others don't. But recently two scholars, working independently, have decisively shown that there is no need to invoke race to explain differences among cultures. Both arrived at that conclusion by eschewing the Standard Social Science Model, in which cultures are arbitrary symbol systems that exist apart from the minds of individual people. In his trilogy Race and Culture, Migrations and Cultures, and Conquests and Cultures, the economist Thomas Sowell explained his starting point for an analysis of cultural differences:
A culture is not a symbolic pattern, preserved like a butterfly in amber. Its place is not in a museum but in the practical activities of daily life, where it evolves under the stress of competing goals and other competing cultures. Cultures do not exist as simply static "differences" to be celebrated but compete with one another as better and worse ways of getting things done -- better and worse, not from the standpoint of some observer, but from the standpoint of the peoples themselves, as they cope and aspire amid the gritty realities of life. 23 {68}
The physiologist Jared Diamond is a proponent of ideas in evolutionary psychology and of consilience between the sciences and the humanities, particularly history. 24 In Guns, Germs, and Steel he rejected the standard assumption that history is just one damn thing after another and tried to explain the sweep of human history over tens of thousands of years in the context of human evolution and ecology. 25 Sowell and Diamond have made an authoritative case that the fates of human societies come neither from chance nor from race but from the human drive to adopt the innovations of others, combined with the vicissitudes of geography and ecology.
Diamond begins at the beginning. For most of human evolutionary history we lived as hunter-gatherers. The
trappings of civilization -- sedentary living, cities, a division of labor, government, professional armies, writing, metallurgy -- sprang from a recent development, farming, about ten thousand years ago. Farming depends on plants and animals that can be tamed and exploited, and only a few species are suited to it. They happened to be concentrated in a few parts of the world, including the Fertile Crescent, China, and Central and South America. The first civilizations arose in those regions.
From then on, geography was destiny. Diamond and Sowell point out that Eurasia, the world's largest landmass, is an enormous catchment area for local innovations. Traders, sojourners, and conquerors can collect them and spread them, and people living at the crossroads can concentrate them into a high-tech package. Also, Eurasia runs in an east- west direction, whereas Africa and the Americas run north-south. Crops and animals that are domesticated in one region can easily be spread to others along lines of latitude, which are also lines of similar climate. But they cannot
be spread as easily along lines of longitude, where a few hundred miles can spell the difference between temperate and tropical climates. Horses domesticated in the Asian steppes, for example, could make their way westward to Europe and eastward to China, but llamas and alpacas domesticated in the Andes never made it northward to Mexico, so the Mayan and Aztec civilizations were left without pack animals. And until recently the transportation of heavy goods over long distances (and with them traders and their ideas) was possible only by water. Europe and parts of Asia are blessed by a notchy, furrowed geography with many natural harbors and navigable rivers. Africa and Australia are not.
? ? ? ? ? ? ? So Eurasia conquered the world not because Eurasians are smarter but because they could best take advantage of the principle that many heads are better than one. The "culture" of any of the conquering nations of Europe, such as Britain, is in fact a greatest-hits collection of inventions assembled across thousands of miles and years. The collection is made up of cereal crops and alphabetic writing from the Middle East, gunpowder and paper from China, domesticated horses from Ukraine, and many others. But the necessarily insular cultures of Australia, Africa, and the Americas had to make do with a few {69} homegrown technologies, and as a result they were no match for their pluralistic conquerors. Even within Eurasia and (later) the Americas, cultures that were isolated by mountainous geography -- for example, in the Appalachians, the Balkans, and the Scottish highlands -- remained backward for centuries in comparison with the vast network of people around them.
The extreme case, Diamond points out, is Tasmania. The Tasmanians, who were nearly exterminated by Europeans in the nineteenth century, were the most technologically primitive people in recorded history. Unlike the Aborigines on the Australian mainland, the Tasmanians had no way of making tire, no boomerangs or spear throwers, no specialized stone tools, no axes with handles, no canoes, no sewing needles, and no ability to fish. Amazingly, the archaeological record shows that their ancestors from the Australian mainland had arrived with these technologies ten thousand years before. But then the land bridge connecting Tasmania to the mainland was submerged and the island was cut off from the rest of the world. Diamond speculates that any technology can be lost from a culture at some point in its history. Perhaps a raw material came to be in short supply and people stopped making the products that depended on it. Perhaps all the skilled artisans in a generation were killed by a freak storm. Perhaps some prehistoric Luddite or ayatollah imposed a taboo on the practice for one inane reason or another. Whenever this happens n a culture that rubs up against other ones, the lost technology can eventually be reacquired as the people clamor for the higher standard of living enjoyed by their neighbors. But in lonely Tasmania, people would have had to reinvent the proverbial wheel every time it was lost, and so their standard of living ratcheted downward.
The ultimate irony of the Standard Social Science Model is that it failed to accomplish the very goal that brought it into being: explaining the different fortunes of human societies without invoking race. The best explanation today is thoroughly cultural, but it depends on seeing a culture as a product of human desires rather than as a shaper of them. History and culture, then, can be grounded in psychology, which can be grounded in computation, neuroscience, genetics, and evolution. But this kind of talk sets off alarms in the minds of many nonscientists. They fear that consilience is a smokescreen for a hostile takeover of the humanities, arts, and social sciences by philistines in white coats. The richness of their subject matter would be dumbed down into a generic palaver about neurons, genes, and evolutionary urges. This scenario is often called "reductionism," and I will conclude the chapter by showing why consilience does not call for it.
Reductionism, like cholesterol, comes in good and bad forms. Bad reductionism -- also called "greedy reductionism" or "destructive reductionism" -- consists of trying to explain a phenomenon in terms of its smallest or simplest
{70} constituents. Greedy reductionism is not a straw man. I know several scientists who believe (or at least say to granting agencies) that we will make break-throughs in education, conflict resolution, and other social concerns by studying the biophysics of neural membranes or the molecular structure of the synapse. But greedy reductionism is far from the majority view, and it is easy to show why it is wrong. As the philosopher Hilary Putnam has pointed out, even the simple fact that a square peg won't fit into a round hole cannot be explained in terms of molecules and atoms but only at a higher level of analysis involving rigidity (regardless of what makes the peg rigid) and geometry. 26 And if anyone really thought that sociology or literature or history could be replaced by biology, why stop there? Biology could in turn be ground up into chemistry, and chemistry into physics, leaving one struggling to explain the causes of World War I in terms of electrons and quarks. Even if World War I consisted of nothing but a very, very large number of quarks in a very, very complicated pattern of motion, no insight is gained by describing it that way.
Good reductionism (also called hierarchical reductionism) consists not of replacing one field of knowledge with another but of connecting or unifying them. The building blocks used by one field are put under a microscope by another. The black boxes get opened; the promissory notes get cashed. A geographer might explain why the coastline of Africa fits into the coastline of the Americas by saying that the landmasses were once adjacent but sat on different plates, which drifted apart. The question of why the plates move gets passed on to the geologists, who appeal to an upwelling of magma that pushes them apart. As for how the magma got so hot, they call in the physicists to explain the reactions in the Earth's core and mantle. None of the scientists is dispensable. An isolated geographer would have to invoke magic to move the continents, and an isolated physicist could not have predicted the shape of South America.
So, too, for the bridge between biology and culture. The big thinkers in the sciences of human nature have been adamant that mental life has to be understood at several levels of analysis, not just the lowest one. The linguist Noam Chomsky, the computational neuroscientist David Marr, and the ethologist Niko Tinbergen have independently marked out a set of levels of analysis for understanding a faculty of the mind. These levels include its function (what
? ? ? it accomplishes in an ultimate, evolutionary sense); its real-time operation (how it works proximately, from moment to moment); how it is implemented in neural tissue; how it develops in the individual; and how it evolved in the species. 27 For example, language is based on a combinatorial grammar designed to communicate an unlimited number of thoughts. It is utilized by people in real time via an interplay of memory lookup and rule application. It is implemented in a network of regions in the center of the left cerebral hemisphere that must coordinate memory, planning, word meaning, and grammar. {71} It develops in the first three years of life in a sequence from babbling to words to word combinations, including errors in which rules maybe overapplied. It evolved through modifications of a vocal tract and brain circuitry that had other uses in earlier primates, because the modifications allowed our ancestors to prosper in a socially interconnected, knowledge-rich lifestyle. None of these levels can be replaced by any of the others, but none can be fully understood in isolation from the others.
Chomsky distinguishes all of these from yet another level of analysis (one that he himself has little use for but that other language scholars invoke). The vantage points I just mentioned treat language as an internal, individual entity, such as the knowledge of Canadian English that I possess in my head. But Language can also be understood as an external entity: the "English language" as a whole, with its fifteen-hundred-year history, its countless dialects and hybrids spanning the globe, its half a million words in the Oxford English Dictionary. An external language is an abstraction that pools the internal languages of hundreds of millions of people living in different places and times. It could not exist without the internal languages in the minds of real humans conversing with one another, but it cannot be reduced to what any of them knows either. For example, the statement "English has a larger vocabulary than Japanese" could be true even if no English speaker has a larger vocabulary than any Japanese speaker.
The English language was shaped by broad historical events that did not take place inside a single head. They include the Scandinavian and Norman invasions in medieval times, which infected it with non-Anglo-Saxon words; the Great Vowel Shift of the fifteenth century, which scrambled the pronunciation of the long vowels and left its spelling system an irregular mess; the expansion of the British Empire, which budded off a variety of Englishes (American, Australian, Singaporean); and the development of global electronic media, which may rehomogenize the language as we all read the same web pages and watch the same television shows.
At the same time, none of these forces can be understood without taking into account the thought processes of flesh- and-blood people. They include the Britons who reanalyzed French words when they absorbed them into English, the children who failed to remember irregular past-tense forms like writhe-wrothe and crow-crew and converted them into regular verbs, the aristocrats who affected fussy pronunciations to differentiate themselves from the rabble, the mumblers who swallowed consonants to leave us made and had (originally maked and haved), and the clever speakers who first converted I had the house built to I had built the house and inadvertently gave English its perfect tense. Language is re-created every generation as it passes through the minds of the humans who speak it. 28
External language is, of course, a fine example of culture, the province of {72} social scientists and scholars in the humanities. The way that language can be understood at some half-dozen connected levels of analysis, from the brain and evolution to the cognitive processes of individuals to vast cultural systems, shows how culture and biology may be connected. The possibilities for connections in other spheres of human knowledge are plentiful, and we will encounter them throughout the book. The moral sense can illuminate legal and ethical codes. The psychology of kinship helps us understand sociopolitical arrangements. The mentality of aggression helps to make sense of war and conflict resolution. Sex differences are relevant to gender politics. Human aesthetics and emotion can enlighten our understanding of the arts.
What is the payoff for connecting the social and cultural levels of analysis to the psychological and biological ones? It is the thrill of discoveries that could never be made within the boundaries of a single discipline, such as universals of beauty, the logic of language, and the components of the moral sense. And it is the uniquely satisfying understanding we have enjoyed from the unification of the other sciences -- the explanation of muscles as tiny magnetic ratchets, of flowers as lures for insects, of the rainbow as a splaying of wavelengths that ordinarily blend into white. It is the difference between stamp collecting and detective work, between slinging around jargon and offering insight, between saying that something just is and explaining why it had to be that way as opposed to some other way it could have been. In a talk-show parody in Monty Python's Flying Circus, an expert on dinosaurs trumpets her new theory of the brontosaurus: "All brontosauruses are thin at one end; much, much thicker in the middle; and then thin again at the far end. " We laugh because she has not explained her subject in terms of deeper principles -- she has not "reduced" it, in the good sense. Even the word understand -- literally, "stand under" -- alludes to descending to a deeper level of analysis.
Our understanding of life has only been enriched by the discovery that living flesh is composed of molecular clockwork rather than quivering protoplasm, or that birds soar by exploiting the laws of physics rather than defying them. In the same way, our understanding of ourselves and our cultures can only be enriched by the discovery that our minds are composed of intricate neural circuits for thinking, feeling, and learning rather than blank slates,
? ? ? ? amorphous blobs, or inscrutable ghosts.
<< {73} >> Chapter 5
The Slate's Last Stand
Human nature is a scientific topic, and as new facts come in, our conception of it will change. Sometimes the facts may show that a theory grants our minds too much innate structure. For example, perhaps our language faculties are equipped not with nouns, verbs, adjectives, and prepositions but only with a distinction between more nounlike and more verblike parts of speech. At other times a theory may turn out to have granted our minds too little innate structure. No current theory of personality can explain why both members of a pair of identical twins reared apart liked to keep rubber bands around their wrists and pretend to sneeze in crowded elevators.
Also up for grabs is exactly how our minds use the information coming in from the senses. Once our faculties for language and social interaction are up and running, some kinds of learning may consist of simply recording information for future use, like the name of a person or the content of a new piece of legislation. Others may be more like setting a dial, flipping a switch, or computing an average, where the apparatus is in place but a parameter is left open so the mind can track variation in the local environment. Still others may use the information provided by all normal environments, such as the presence of gravity or the statistics of colors and lines in the visual field, to tune up our sensorimotor systems. There are yet other ways that nature and nurture might interact, and many will blur the distinction between the two.
This book is based on the estimation that whatever the exact picture turns out to be, a universal complex human nature will be part of it. I think we have reason to believe that the mind is equipped with a battery of emotions, drives, and faculties for reasoning and communicating, and that they have a common logic across cultures, are difficult to erase or redesign from scratch, were shaped by natural selection acting over the course of human evolution, and owe some of their basic design (and some of their variation) to information in the genome. This general picture is meant to embrace a variety of theories, present and future, and a range of foreseeable scientific discoveries. {74}
But the picture does not embrace just any theory or discovery. Conceivably scientists might discover that there is insufficient information in the genome to specify any innate circuitry, or no known mechanism by which it could be wired into the brain. Or perhaps they will discover that brains are made out of general-purpose stuff that can soak up just about any pattern in the sensory input and organize itself to accomplish just about any goal. The former discovery would make innate organization impossible; the latter would make it unnecessary. Those discoveries would call into question the very concept of human nature. Unlike the moral and political objections to the concept of human nature (objections that I discuss in the rest of this book), these would be scientific objections. If such discoveries are on the horizon, I had better look at them carefully.
This chapter is about three scientific developments that are sometimes interpreted as undermining the possibility of a complex human nature. The first comes from the Human Genome Project. When the sequence of the human genome was published in 2001, geneticists were surprised that the number of genes was lower than they had predicted. The estimates hovered around 34,000 genes, which lies well outside the earlier range of 50,000 to 100,000. 1 Some editorialists concluded that the smaller gene count refuted any claim about innate talents or tendencies, because the slate is too small to contain much writing. Some even saw it as vindicating the concept of free will: the smaller the machine, the more room for a ghost.
The second challenge comes from the use of computer models of neural networks to explain cognitive processes. These artificial neural networks can often be quite good at learning statistical patterns in their input. Some modelers from the school of cognitive science called connectionism suggest that generic neural networks can account for all of human cognition, with little or no innate tailoring for particular faculties such as social reasoning or language. In Chapter 2 we met the founders of connectionism, David Rumelhart and James McClelland, who suggested that people are smarter than rats only because they have more associative cortex and because their environment contains a culture to organize it.
The third comes from the study of neural plasticity, which examines how the brain develops in the womb and early childhood and how it records experience as the animal learns. Neuroscientists have recently shown how the brain changes in response to learning, practice, and input from the senses. One spin on these discoveries may be called extreme plasticity. According to this slant, the cerebral cortex -- the convoluted gray matter responsible for perception, thinking, language, and memory -- is a protean substance that can be shaped almost limitlessly by the structure and demands of the environment. The blank slate becomes the plastic slate.
? ? ? ? ? ? ? ? Connectionism and extreme plasticity are popular among cognitive {75} scientists at the West Pole, who reject a completely blank slate but want to restrict innate organization to simple biases in attention and memory. Extreme plasticity also appeals to neuroscientists who wish to boost the importance of their field for education and social policy, and to entrepreneurs selling products to speed up infant development, cure learning disabilities, or slow down aging. Outside the sciences, all three developments have been welcomed by some scholars in the humanities who want to beat back the encroachments of biology. 2 The lean genome, connectionism, and extreme plasticity are the Blank Slate's last stand.
The point of this chapter is that these claims are not vindications of the doctrine of the Blank Slate but products of the Blank Slate. Many people (including a few scientists) have selectively read the evidence, sometimes in bizarre ways, to fit with a prior belief that the mind cannot possibly have any innate structure, or with simplistic notions of how innate structure, if it did exist, would be encoded in the genes and develop in the brain.
I should say at the outset that I find these latest-and-best blank-slate theories highly implausible -- indeed, barely coherent. Nothing comes out of nothing, and the complexity of the brain has to come from somewhere. It cannot come from the environment alone, because the whole point of having a brain is to accomplish certain goals, and the environment has no idea what those goals are. A given environment can accommodate organisms that build dams, migrate by the stars, trill and twitter to impress the females, scent-mark trees, write sonnets, and so on. To one species, a snatch of human speech is a warning to flee; to another, it is an interesting new sound to incorporate into its own vocal repertoire; to a third, it is grist for grammatical analysis. Information in the world doesn't tell you what to do with it.
Also, brain tissue is not some genie that can grant its owner any power that would come in handy. It is a physical mechanism, an arrangement of matter that converts inputs to outputs in particular ways. The idea that a single generic substance can see in depth, control the hands, attract a mate, bring up children, elude predators, outsmart prey, and so on, without some degree of specialization, is not credible. Saying that the brain solves these problems because of its "plasticity" is not much better than saying it solves them by magic. Still, in this chapter I will examine the latest scientific objections to human nature carefully. Each of the discoveries is important on its own terms, even if it does not support the extravagant conclusions that have been drawn. And once the last supports for the Blank Slate have been evaluated, I can properly sum up the scientific case for the alternative.
The human genome is often seen as the essence of our species, so it is not surprising that when its sequence was announced in 2001 commentators rushed to give it the correct interpretation for human affairs. Craig Venter, {76} whose company had competed with a public consortium in the race to sequence the genome, said at a press conference that the smaller-than-expected gene count shows that "we simply do not have enough genes for this idea of biological determinism to be right. The wonderful diversity of the human species is not hard-wired in our genetic code. Our environments are critical. " In the United Kingdom, The Guardian headlined its story, "Revealed: The Secret of Human Behaviour. Environment, Not Genes, Key to Our Acts. "3 An editorial in another British newspaper concluded that "we are more free, it seems, than we had realized. " Moreover, the finding "offers comfort for the left, with its belief in the potential of all, however deprived their background. But it is damning for the right, with its fondness for ruling classes and original sin. "4
All this from the number 34,000! Which leads to the question, What number of genes would have proven that the diversity of our species was wired into our genetic code, or that we are less free than we had realized, or that the political right is right and the left is wrong? 50,000? 150,000? Conversely, if it turned out that we had only 20,000 genes, would that have made us even freer, or the environment even more important, or the political left even more comfortable? The fact is that no one knows what these numbers mean. No one has the slightest idea how many genes it would take to build a system of hard-wired modules, or a general-purpose learning program, or anything in between -- to say nothing of original sin or the superiority of the ruling class.
In our current state of ignorance of how the genes build a brain, the number of genes in the human genome is just a number.
If you don't believe this, consider the roundworm Caenorhabditis elegans, which has about 18,000 genes. By the logic of the genome editorialists, it should be twice as free, be twice as diverse, and have twice as much potential as a human being. In fact, it is a microscopic worm composed of 959 cells grown by a rigid genetic program, with a nervous system consisting of exactly 302 neurons in a fixed wiring diagram. As far as behavior is concerned, it eats, mates, approaches and avoids certain smells, and that's about it. This alone should make it obvious that our freedom and diversity of behavior come from having a complex biological makeup, not a simple one.
Now, it is a genuine puzzle why humans, with their hundred trillion cells and hundred billion neurons, need only twice as many genes as a humble little worm. Many biologists believe that the human genes have been undercounted. The number of genes in a genome can only be estimated; right now they cannot literally be totted up. Gene- estimating programs look for sequences in the DNA that are similar to known genes and that are active enough to be caught in the act of building a protein. 5 Genes that are unique to humans or active only in the developing brain of the
? ? ? ? ? ? fetus -- the genes most relevant to human nature -- and other inconspicuous genes could evade the software and get left {77} out of the estimates. Alternative estimates of 57,000,75,000, and even 120,000 human genes are currently being bruited about. 6 Still, even if humans had six times as many genes as a roundworm rather than just twice as many, the puzzle would remain.
Most biologists who are pondering the puzzle don't conclude that humans are less complex than we thought. Instead they conclude that the number of genes in a genome has little to do with the complexity of the organism. 7 A single
gene does not correspond to a single component in such a way that an organism with 20,000 genes has 20,000 components, an organism with 30,000 genes has 30,000 components, and so on. Genes specify proteins, and some of the proteins do become the meat and juices of an organism. But other proteins turn genes on or off, speed up or slow down their activity, or cut and splice other proteins into new combinations. James Watson points out that we should recalibrate our intuitions about what a given number of genes can do: "Imagine watching a play with thirty thousand actors. You'd get pretty conrused. "
Depending on how the genes interact, the assembly process can be much more intricate for one organism than for another with the same number of genes. In a simple organism, many of the genes simply build a protein and dump it into the stew. In a complex organism, one gene may turn on a second one, which speeds up the activity of a third one (but only if a fourth one is active), which then turns off the original gene (but only if a fifth one is inactive), and so on. This defines a kind of recipe that can build a more complex organism out of the same number of genes. The complexity of an organism thus depends not just on its gene count but on the intricacy of the box-and-arrow diagram that captures how each gene impinges on the activity of the other genes. 8 And because adding a gene doesn't just add an ingredient but can multiply the number of ways that the genes can interact with one another, the complexity of organisms depends on the number of possible combinations of active and inactive genes in their genomes. The geneticist Jean-Michel Claverie suggests that it might be estimated by the number two (active versus inactive) raised to the power of the number of genes. By that measure, a human genome is not twice as complex as a roundworm genome but 216,000 (a one followed by 4. 800 zeroes) times as complex. 9
There are two other reasons why the complexity of the genome is not reflected in the number of genes it contains. One is that a given gene can produce not just one protein but several. A gene is typically broken into stretches of DNA that code for fragments of protein (exons) separated by stretches of DNA that don't (introns), a bit like a magazine article interrupted by ads. The segments of a gene can then be spliced together in multiple ways. A gene composed of exons A, B, C, and D might give rise to proteins corresponding to {78} ABC, ABD, ACD, and so on -- as many as ten different proteins per gene. This happens to a greater degree in complex organisms than in simple ones. 10
Second, the 34,000 genes take up only about 3 percent of the human genome. The rest consists of DNA that does not code for protein and that used to be dismissed as "junk. " But as one biologist recently put it, "The term 'junk DNA' is a reflection of our ignorance. "11 The size, placement, and content of the noncoding DNA can have dramatic effects on the way that nearby genes are activated to make proteins. Information in the billions of bases in the non-coding regions of the genome is part of the specification of a human being, above and beyond the information contained in the 34,000 genes.
The human genome, then, is fully capable of building a complex brain, in spite of the bizarre proclamations of how wonderful it is that people are almost as simple as worms. Of course "the wonderful diversity of the human species is not hard-wired in our genetic code," but we didn't need to count genes to figure that out -- we already know it from the fact that a child growing up in Japan speaks Japanese but the same child growing up in England would speak English. It is an example of a syndrome we will meet elsewhere in this book: scientific findings spin-doctored beyond recognition to make a moral point that could have been made more easily on other grounds.
The second scientific defense of the Blank Slate comes from connectionism, the theory that the brain is like the artificial neural networks simulated on computers to learn statistical patterns. 12
Cognitive scientists agree that the elementary processes that make up the instruction set of the brain -- storing and retrieving an association, sequencing elements, focusing attention -- are implemented in the brain as networks of densely interconnected neurons (brain cells). The question is whether a generic kind of network, after being shaped by the environment, can explain all of human psychology, or whether the genome tailors different networks to the demands of particular domains: language, vision, morality, fear, lust, intuitive psychology, and so on. The connectionists, of course, do not believe in a blank slate, but they do believe in the closest mechanistic equivalent, a general-purpose learning device.
What is a neural network? Connectionists use the term to refer not to real neural circuitry in the brain but to a kind of computer program based on the metaphor of neurons and neural circuits. In the most common approach, a "neuron" carries information by being more or less active. The activity level indicates the presence or absence (or intensity or degree of confidence) of a simple feature of the world. The feature may be a color, a line with a certain slant, a letter
? ? ? ? ? ? ? ? ? of the alphabet, or a property of an animal such as having four legs.
A network of neurons can represent different concepts, depending on {79} which ones are active. If neurons for "yellow," "flies," and "sings" are active, the network is thinking about a canary; if neurons for "silver," "flies," and "roars" are active, it is thinking about an airplane. An artificial neural network computes in the following manner. Neurons are linked to other neurons by connections that work something like synapses. Each neuron counts up the inputs from other neurons and changes its activity level in response. The network learns by allowing the input to change the strengths of the connections. The strength of a connection determines the likelihood that the input neuron will excite or inhibit the output neuron.
Depending on what the neurons stand for, how they are innately wired, and how the connections change with training, a connectionist network can learn to compute various things. If everything is connected to everything else, a network can soak up the correlations among features in a set of objects. For example, after exposure to descriptions of many birds it can predict that feathered singing things tend to fly or that feathered flying things tend to sing or that singing flying things tend to have feathers. If a network has an input layer connected to an output layer, it can learn associations between ideas, such as that small soft flying things are animals but large metallic flying things are vehicles. If its output layer feeds back to earlier layers, it can crank out ordered sequences, such as the sounds making up a word.
The appeal of neural networks is that they automatically generalize their training to similar new items. If a network has been trained that tigers eat Frosted Flakes, it will tend to generalize that lions eat Frosted Flakes, because eating Frosted Flakes" has been associated not with "tigers" but with simpler features like "roars" and "has whiskers," which make up part of the representation of h'ons, too. The school of connectionism, like the school of associationism championed by Locke, Hume, and Mill, asserts that these generalizations are the crux of intelligence. If so, highly trained but otherwise generic neural networks can explain intelligence.
Computer modelers often set their models on simplified toy problems to ? rove that they can work in principle. The question then becomes whether the models can "scale up" to more realistic problems, or whether, as skeptics say, the modeler "is climbing trees to get to the moon. " Here we have the problem with connectionism. Simple connectionist networks can manage impressive displays of memory and generalization in circumscribed problems like reading a list of words or learning stereotypes of animals. But they are simply too underpowered to duplicate more realistic feats of human intelligence like understanding a sentence or reasoning about living things.
Humans don't just loosely associate things that resemble each other, or things that tend to occur together. They have combinatorial minds that enterrain propositions about what is true of what, and about who did what to {80} whom, when and where and why. And that requires a computational architecture that is more sophisticated than the uniform tangle of neurons used in generic connectionist networks. It requires an architecture equipped with logical apparatus like rules, variables, propositions, goal states, and different kinds of data structures, organized into larger systems. Many cognitive scientists have made this point, including Gary Marcus, Marvin Minsky, Seymour Papert, Jerry Fodor, Zenon Pylyshyn, John Anderson, Tom Bever, and Robert Hadley, and it is acknowledged as well by neural network modelers who are not in the connectionist school, such as John Hummel, Lokendra Shastri, and Paul Smolensky. 13 I have written at length on the limits of connectionism, both in scholarly papers and in popular books; here is a summary of my own case. 14
In a section called "Connectoplasm" in How the Mind Works, I laid out some simple logical relationships that underlie our understanding of a complete thought (such as the meaning of a sentence) but that are difficult to represent in generic networks. 15 One is the distinction between a kind and an individual: between ducks in general and this duck in particular. Both have the same features (swims, quacks, has feathers, and so on), and both are thus represented by the same set of active units in a standard connectionist model. But people know the difference.
A second talent is compositionality: the ability to entertain a new, complex thought that is not just the sum of the simple thoughts composing it but depends on their relationships. The thought that cats chase mice, for example, cannot be captured by activating a unit each for "cats," "mice," and "chase," because that pattern could just as easily stand for mice chasing cats.
A third logical talent is quantification (or the binding of variables): the difference between fooling some of the people all of the time and fooling all of the people some of the time. Without the computational equivalent of x's,y's, parentheses, and statements like "For all x," a model cannot tell the difference.
A fourth is recursion: the ability to embed one thought inside another, so that we can entertain not only the thought that Elvis lives, but the thought that the National Enquirer reported that Elvis lives, that some people believe the National Enquirer report that Elvis lives, that it is amazing that some people believe the National Enquirer report that Elvis lives, and so on. Connectionist networks would superimpose these propositions and thereby confuse their various subjects and predicates.
A final elusive talent is our ability to engage in categorical, as opposed to fuzzy, reasoning: to understand that Bob
? ? ? ? ? Dylan is a grandfather, even though he is not very grandfatherly, or that shrews are not rodents, though they look just like mice. With nothing but a soup of neurons to stand for an object's properties, and no provision for rules, variables, and definitions, the networks fall back on stereotypes and are bamboozled by atypical examples. {81}
In Words and Rules I aimed a microscope on a single phenomenon of language that has served as a test case for the ability of generic associative networks to account for the essence of language: assembling words, or pieces of words, into new combinations. People don't just memorize snatches of language but create new ones. A simple example is the English past tense. Given a neologism like to spam or to snarf, people don't have to run to the dictionary to look up their past-tense forms; they instinctively know that they are spammed and snarfed. The talent for assembling new combinations appears as early as age two, when children overapply the past-tense suffix to irregular verbs, as in We holded the baby rabbits and Horton heared a Who. 16
The obvious way to explain this talent is to appeal to two kinds of computational operations in the mind. Irregular forms like held and heard are stored in and retrieved from memory, just like any other word. Regular forms like walk- walked can be generated by a mental version of the grammatical rule "Add -ed to the verb. " The rule can apply whenever memory fails. It may be used when a word is unfamiliar and no past-tense form had been stored in memory, as in to spam, and it may be used by children when they cannot recall an irregular form like heard and need some way of marking its tense. Combining a suffix with a verb is a small example of an important human talent: combining words and phrases to create new sentences and thereby express new thoughts. It is one of the new ideas of the cognitive revolution introduced in Chapter 3, and one of the logical challenges for connectionism I listed in the preceding discussion.
Connectionists have used the past tense as a proving ground to see if they could duplicate this textbook example of human creativity without using a rule and without dividing the labor between a system for memory and a system for grammatical combination. A series of computer models have tried to generate past-tense forms using simple pattern associator networks. The networks typically connect the sounds in verbs with the sounds in the past-tense form: -am with -ammed, -ing with -ung, and so on. The models can then generate new forms by analogy, just like the generalization from tigers to lions: trained on crammed, a model can guess spammed; trained on folded, it tends to say holded.
But human speakers do far more than associate sounds with sounds, and the models thus fail to do them justice. The failures come from the absence of machinery to handle logical relationships. Most of the models are baffled by new words that sound different from familiar words and hence cannot be generalized by analogy. Given the novel verb to frilg, for example, they come up not with frilged, as people do, but with an odd mishmash like freezled. That is because they lack the device of a variable, like x in algebra or "verb" in grammar, which can apply to any member of a category, regardless of how familiar {82} its properties are. (This is the gadget that allows people to engage in categorical rather than fuzzy reasoning. ) The networks can only associate bits of sound with bits of sound, so when confronted with a new verb that does not sound like anything they were trained on, they assemble a pastiche of the most similar sounds they can find in their network.
The models also cannot properly distinguish among verbs that have the same sounds but different past-tense forms, such as ring the bell-rang the bell and ring the city-ringed the city. That is because the standard models represent
only sound and are blind to the grammatical differences among verbs that call for different conjugations. The key difference here is between simple roots like ring in the sense of "resonate" (past tense rang) and complex verbs derived from nouns like ring in the sense of "form a ring around" (past tense ringed). To register that difference, a language-using system has to be equipped with compositional data structures (such as "a verb made from the noun ring") and not just a beanbag of units.
Yet another problem is that connectionist networks track the statistics of the input closely: how many verbs of each sound pattern they have encountered. That leaves them unable to account for the epiphany in which young children discover the -ed rule and start making errors like holded and heared. Connectionist modelers can induce these errors only by bombarding the network with regular verbs (so as to burn in the -ed) in a way that is unlike anything real children experience. Finally, a mass of evidence from cognitive neuroscience shows that grammatical combination (including regular verbs) and lexical lookup (including irregular verbs) are handled by different systems in the brain rather than by a single associative network.
It's not that neural networks are incapable of handling the meanings of sentences or the task of grammatical conjugation. (They had better not be, since the very idea that thinking is a form of neural computation requires that some kind of neural network duplicate whatever the mind can do. ) The problem lies in the credo that one can do everything with a generic model as long as it is sufficiently trained. Many modelers have beefed up, retrofitted, or combined networks into more complicated and powerful systems. They have dedicated hunks of neural hardware to abstract symbols like "verb phrase" and "proposition" and have implemented additional mechanisms (such as synchronized firing patterns) to bind them together in the equivalent of compositional, recursive symbol structures.
? ? ? ? They have installed banks of neurons for words, or for English suffixes, or for key grammatical distinctions. They have built hybrid systems, with one network that retrieves irregular forms from memory and another that combines a verb with a suffix. 17
A system assembled out of beefed-up subnetworks could escape all the criticisms. But then we would no longer be talking about a generic neural network! We would be talking about a complex system innately tailored to {83} compute a task that people are good at. In the children's story called "Stone Soup," a hobo borrows the use of a woman's kitchen ostensibly to make soup from a stone. But he gradually asks for more and more ingredients to balance the flavor until he has prepared a rich and hearty stew at her expense. Connectionist modelers who claim to build intelligence out of generic neural networks without requiring anything innate are engaged in a similar business. The design choices that make a neural network system smart -- what each of the neurons represents, how they are wired together, what kinds of networks are assembled into a bigger system, in which way -- embody the innate organization of the part of the mind being modeled. They are typically hand-picked by the modeler, like an inventor rummaging through a box of transistors and diodes, but in a real brain they would have evolved by natural selection (indeed, in some networks, the architecture of the model does evolve by a simulation of natural selection). 18 The only alternative is that some previous episode of learning left the networks in a state ready for the current learning, but of course the buck has to stop at some innate specification of the first networks that kick off the learning process.
So the rumor that neural networks can replace mental structure with statistical learning is not true. Simple, generic networks are not up to the demands of ordinary human thinking and speaking; complex, specialized networks are a stone soup in which much of the interesting work has been done in setting up the innate wiring of the network. Once this is recognized, neural network modeling becomes an indispensable complement to the theory of a complex human nature rather than a replacement for it. 19 It bridges the gap between the elementary steps of cognition and the physiological activity of the brain and thus serves as an important link in the long chain of explanation between biology and culture. ~
FOR MOST OF its history, neuroscience was faced with an embarrassment: the brain looked as if it were innately specified in every detail. When it comes to the body, we can see many of the effects of a person's life experience: it may be tanned or pale, callused or soft, scrawny or plump or chiseled. But no such marks could be found in the brain. Now, something has to be wrong with this picture. People learn, and learn massively: they learn their language, their culture, their know-how, their database of facts. Also, the hundred trillion connections in the brain cannot possibly be specified individually by a 750-megabyte genome. The brain somehow must change in response to its input; the only question is how.
We are finally beginning to understand how. The study of neural plasticity is hot. Almost every week sees a discovery about how the brain gets wired in the womb and tuned outside it. After all those decades in which no one could find anything that changed in the brain, it is not surprising that the {84} discovery of plasticity has given the nature-nurture pendulum a push. Some people describe plasticity as a harbinger of an expansion of human potential in which the powers of the brain will be harnessed to revolutionize childrearing, education, therapy, and aging. And several manifestos have proclaimed that plasticity proves that the brain cannot have any significant innate organization. 20 In Rethinking Innateness, Jeffrey Elman and a team of West Pole connectionists write that predispositions to think about different things in different ways (language, people, objects, and so on) may be implemented in the brain only as "attention-grabbers" that ensure that the organism will receive "massive experience of certain inputs prior to subsequent learning. "21 In a "constructivist manifesto," the theoretical neuroscientists Stephen Quartz and Terrence Sejnowski write that "although the cortex is not a tabula rasa . . . it is largely equipotential at early stages," and therefore that innatist theories "appear implausible. "22
Neural development and plasticity unquestionably make up one of the great frontiers of human knowledge. How a linear string of DNA can direct the assembly of an intricate three-dimensional organ that lets us think, feel, and learn is a problem to stagger the imagination, to keep neuroscientists engaged for decades, and to belie any suggestion that we are approaching "the end of science. "
And the discoveries themselves are fascinating and provocative. The cerebral cortex (outer gray matter) of the brain has long been known to be divided into areas with different functions. Some represent particular body parts; others represent the visual field or the world of sound; still others concentrate on aspects of language or thinking. We now know that with learning and practice some of their boundaries can move around. (This does not mean that the brain tissue literally grows or shrinks, only that if the cortex is probed with electrodes or monitored with a scanner, the boundary where one ability leaves off and the next one begins can shift. ) Violinists, for example, have an expanded region of cortex representing the fingers of the left hand. 23 If a person or a monkey is trained on a simple task like recognizing shapes or attending to a location in space, neuroscientists can watch as parts of the cortex, or even
? ? ? ? ? ? ? ?