TO CATCH A THIEF (USING INFERENCE SYSTEMS)
This description may give the impression that sorting objects along ontological distinctions and producing category-specific inferences is a matter of explicit, deliberate thinking. Far from it. The distinctions are constantly produced by the mind. We do not have to think about them. To get a sense of how smoothly inference systems work, imagine the following scene:
In a quiet and prosperous suburb, a dapper old gentleman with a hat comes out the back door of a house and walks across the lawn. He is carrying a big screwdriver and a crowbar, which he puts in his trousers' side-pockets. He looks around a few times and then proceeds along the pavement. Not far from there, a child is playing with a huge Labrador on a leash. All of a sudden, the dog starts at the sight of a cat in the next garden and gives a sudden pull that makes the leash snap out of the child's hand. The dog dashes after its prey, charges across the pavement and knocks over the old man, who trips and falls flat on his face, his hat rolling in the gutter. The man yells in pain as the screwdriver has sprung out of his pocket and badly cut his arm. The man picks himself up and limps away, massaging his bloodied hand, leaving his hat in the gutter. You were not the only witness of all this; a police officer was patrolling the neighborhood. She picks up the hat, runs after the gentleman, puts her hand on his shoulder and says "Hey, wait!" As the man turns he recoils in visible shock at the sight of the police officer, looks around as if trying to find an escape route and finally says: "All right, all right. It's a fair cop." From his pockets he extracts a handfnl of rings and necklaces and hands them over to the bemused police officer.
The scene illustrates how multiple inference systems are involved in the perception of apparently simple events. Were you a witness to all this you might be surprised by some events but you would understand all of them. This is not because there is some center in the brain that is busy understanding "what is happening to the man, the little girl, the dog and the police officer." It is because a whole confederacy of different systems are involved in handling particular aspects of the scene. Consider these:
• Understanding the pysics of solid objects: The dog yanked the leash out of the child's hand and knocked over a passerby. The link between the leash and the dog's collar is stronger than the child's grip on the leash, and the man is knocked aside because both he and the dog are solid objects that collide when their trajectories cross. This kind of phenomenon is automatically represented in our minds by a set of mechanisms that do what psychologists call "intuitive physics," in analogy with scientific physics.
• Understanding physical causation: In the scene you witnessed, you saw the dog hit the man on its way and you then saw the man stumble and fall down. But that is not the way you would describe it. What seems to have happened is that the man tripped because he had been hit by the charging dog. Physical events around us are not just one damn thing after another; there often appear to be causes and effects. But you cannot see a cause, at least literally. What you see are events and your brain interprets their succession as cause plus effect.
• Detecting goal-directed motion: The dog charged across the street in a particular direction that happened to point toward the cat's location. To put things in a more natural way, the dog's goal was to get closer to the cat. If all you saw was physical motion, you would think that some invisible force was driving the dog toward the cat. But an inference system in your mind suggests that this invisible force is inside the dog's head, in his desire to get closer to something that looks like prey.
• Keeping track of who's who: The scene makes sense to you as an eyewitness only if you can track the different characters and keep a particular "file" on each of them with an account of what just happened to them or what they just did. This seems of course trivially easy, if some system in your brain takes a snapshot of every character's face, and then manages to reidentify the different characters, even though faces and bodies change orientation, they are partly occluded, the lighting is different, etc.
• Linking structure to function: The screwdriver hurt the man as he fell down. This is not too surprising as this instrument was probably hard, pointed and the blade at the end was probably sharp. We intuitively guess all this, not just because scrowdrivers in general are like that but also because there is a reason for these features: they help in performing particular functions. That we expect tools to have such functional features is manifest in the surprise that would be created if the crowbar or screwdriver happened to be soft as rubber.
• Understanding mental representation: This too is indispensable if we are to make sense of what happened to the thief and the police officer. To us witnesses of what happened, she saw that the man had dropped his hat and wanted to give it back to him. He thought she knew he had broken into a house. But she in fact did not know that, although she immediately deduced what was going on when she saw the jewels. She also realized the thief had not understood that she only wanted to help him.... One could go on for some time. The point is that you cannot understand the story if you do not maintain a rather sophisticated account of who thinks what about whom. But thoughts are invisible. You cannot observe them directly, you have to infer them.
This is not, by far, a complete list of all the systems engaged, but it should be enough to give an idea of what I want to emphasize here. The most banal scenes of everyday life are replete with facts that seem obvious or simple only because we have a huge mental basement filled with extremely efficient servants, whose activities are not available for detailed conscious inspection. Each of these specialized systems only handles a limited aspect of the information available about our surroundings but produces very smart inferences about that aspect. This is why all these systems in the brain are called inference systems.
This is where scientific discoveries go against the grain of common sense. We may think that there is nothing terribly complicated in understanding, for instance, how objects move when they are pushed, what happens when they collide, why an object will fall if there is nothing underneath to support itin other words what psychologists call "intuitive physics." If I drop an object, you expect it to fall downward with a vertical trajectory. If I throw a ball against a wall, you expect it to bounce at an angle that is roughly symmetrical to that at which it hit the wall. If a billiard ball is on the path of another billiard ball, you expect them to collide, not go through one another. If you throw a tennis ball as hard as you can, you expect it to fly higher and faster than if you just gave it a gentle nudge. Intuitive physics, like its scientific counterpart, is based on principles. These principles take as input a particular description of what objects are around and what their motion is, and produce expectations about the next step. That we have precise expectations is not something we are aware of. It is made manifest only when some aspect of physical reality around us violates the principles which is why experimental psychologists often use cheap magic tricks to produce counterintuitive situations.
Intuitive physics uses observable phenomena (like the motion of objects) to infer what is intrinsically invisible. Consider for instance causal connections between events. If you see a billiard ball hitting another one, you can't but perceive that the second ball moved because it was hit by the first one. Indeed, we sometimes think we "see" such events even when we know that no physical object is involved. If you show people colored discs moving on a screen, you can make them "perceive" that one disc "hit" another one, "pushed" it along, and so on. This happens even if people know that what's on the screen are just dots of light, so that there is nothing that is hitting or pushing anything. Now if you adjust the displays a bit, you can make the "causal illusion" dissppear. What people see now are discs moving, other discs moving too, but no "cause" and no "effect." In the 1940s, psychologists Albert Michotte and Fritz Heider showed that such "causal illusions" are very reliableeverybody reports that they "saw" a causeand that they depend on precise mathematical relations ta between the motions of the objectsyou can eliminate the illusion or re-create it by adjusting the displays according to precise formulae. More surprising, Heider and Michotte had also shown that dots on a screen can make people believe that what they see are not jost solid objects moving and colliding but animate beings "chasing" or "avoiding" each other. Again, by carefally adjusting the timing and spatial characteristics of the dots' relative motion, you can create the illusion of "social causation." In the same way as in the simpler experiments, the adult subjects know that what they see are in fact just dots on a screen. But they cannot help perceive them as engaged in a "chase" or as "trying to get somewhere."
In the previous chapter I gave a very simplified account of what it is to have ontological categories. I suggested that we have a mental catalogue of the kinds of things that are around us, containing entries like "animal," "person" and "man-made object," and a little theory about each entry. The theory specifies for instance that animals are born of animals of the same species, that the structure of man-made objects is related to their use, etc. But the term theory may be a bit misleading, so here is a more precise description. Seeing or otherwise perceiving an object activates a particular set of inference systems. Not all objects activate them all. The fact that a certain type of object activates a certain panoply of inference systems is what we mean when we say that it belongs to a particular category.
To return to the thief and the police officer: you formed certain expectations about the physics of the dog and the old man. So when their trajectories coincided you were not surprised that they collided (rather than going straight through one another). So we can say that looking at the dog and the man had activated your intnitive physics system. This system is activated also when you look at inert objects like trees or at man-made objects. But the dog and the man and the police officer also activated your goal-detection system, which is why you could spontaneously assume that the dog was trying to catch the cat, the man was trying to avoid the dog, and the police officer was trying to catch up with the man. These persons also activate a more complicated intuitive psychology system, which produces subtle inferences such as "she realized that he had not realized that she did not know what he was up to"a description that would never be produced if you were considering a tree, and might be produced in a much simpler form if you were considering a mouse or a worm. That the screwdriver was hard and sharp would be an expectation of your structure-function system. This system is also activated by animal or human body parts: seeing a cat's claws you immediately expect them to be there so that the animal can rip open its prey's body. On the other hand, when you see a tool, you immediately activate a description not only of its functional features but also of its use by a human handfor instance the fact that a screwdriver or gimlet is designed to be turned whereas a crowbar is meant to be pressed down on.
All this to show that you can replace what I called "ontological categories" with "theories", with a list of appropriate inference systems. If something activates physics, goal-detection, as well as some biological expectations I will describe below, then it is what we usually call an "animal." If it activates all that plus intuitive psychology, it is what we usually call a "person." If it activates physics and structure-function, it may be either a "man-made object" or an "animal part." If in addition it activates intentional use, it is what we usually call a "tool." Instead of having a complex mental encyclopedia with theoretical declarations about what animals and artifacts and persons are, all we have are flags that switch on particular systems and turn other systems off.
Our knowledge of inference systems has recently made huge progress because of independent findings in four different fields. Experimental studies of normal adult subjects showed how their intuitions about various aspects of their environwent are based on specialized principles. Causation is one among many examples of such principles, as we will see presently. Also, the study of cognitive development has shown with much more precision than before how some of these principles appear very early in infancy and how they make it possible to acquire vast amounts of knowledge so quickly. At the same time, imagery techniques that track blood flow or electrical and magnetic activity in the brain have reached a suffficient level of precision to tell us which parts of the cortex and other brain structures are active in different kinds of tasks. Finally, neuropsychologists have discovered a whole set of cognitive pathologies that impair some inference systems while leaving the rest intact, which suggests how the system is organized.
WHEELS WITHIN WHEELS: SYSTEMS IN THE BRAIN
In this model, then, what makes our minds smart is not really a set of encyclopedic descriptions of such things as artifacts and animals in general but the fact that very specialized systems are selectively turned on or off when we consider different kinds of objects. This description is better than the previous one for several reasons. First, it makes sense of the fact that some inference systems are activated by several different kinds of objects. Goal-detection is applied to dogs and to persons. Structure-function is applied both to artifacts and to some body parts. Also, the way I talked of "ontological categories" as if these were real kinds of things in the world was misleading because many objects migrate from one of these so-called categories to another, depending on the way we consider them.
A description of our minds as a bundle of inference systems, differently activated by different objects, is better than that of a mental encyclopedia because it is much closer to the way a brain is actually organized. That is, there is no general "catalogue of all things" in the brain with their different characteristics; nor is there a division in the brain between the bits that deal with animals, those that deal with persons, those that only consider artifacts, etc. Instead, there are many different functional systems that work to produce particular kinds of inferences about different aspects of our surroundings. This is not just theoretical speculation: that there are different systems, and that they are narrow specialists, is made manifest both by neuro-imaging and by pathology.
Consider for instance the domain of man-made objects. This would seem to be a straightforward ontological category. Many objects in our world were made by people and many were not. If our brain had been designed by philosophers, it would certainly differentiate between man-made and non-man-made stuff in general. But the brain is subtler than that, because it was designed by evolution. When people are presented novel artifact-like and animal-like pictures, their brains do show different activation. In the case of artifacts, there seems to be enough activity in the pre-motor cortex (involved in planning movements) to suggest that the system is trying to figure out (forgive the anthropomorphic tone: the system is of course not aware of what it is doing) some way of handling this new object. But this only applies if the object is tool-like. In other words, there may not be a category of "artifacts" in the brain, but there is a system for "finding out how to handle tool-like objects," which is far more specific.
The specificity is even more obvious in the handling of complex domains such as animacy and intentionality. In my interpretation of the story above, I simplified matters a great deal when I said that we have a system that computes mental states like knowing, hoping, perceiving, inferring, etc., and produces descriptions of these states in other people's minds, as an explanation for (and prediction of) their behavior. My description was simplified in that this intuitive psychological system is in fact composed of a variety of subsystems. The whole scene with the thief - especially the spectacular denouement - made sense only because you grasped some aspects of what was going on in these different people's minds. This was made possible by specialized mechanisms that constantly produce representations of what is going on inside people's heads, in terms of perceptions, intentions, beliefs, etc.
That this requires subtle and specialized machinery is made obvious, indeed spectacularly so, by the fact that a part of this machinery is impaired in some people. They can compute the trajectories of solid objects and their causal connections, predict where things will fall, identify different persons, etc., but the simplest psychological processes escape them. Indeed, the story of the thief and the police offficer is largely inspired by similar anecdotes used by neuropsychologist Chris Frith and his colleagues to test autistic patients. Children and adults with this condition have great difficulty making sense of such anecdotes. Why the man would give the police offficer the jewels and why she would be surprised are events that they register but cannot easily explain. Also, Frith showed that a specific pattern of brain activation occurs when normal subjects listen to such a story. This activation is typical of what happens when they have to represent how other people represented a certain scene. But the autistic have a rather different pattern of activation, which would indicate that their "theory of mind" mechanism is either not functioning or functioning in a very different way.
This interpretation of autism as a failure to represent other people's representations was originally proposed by three developmental psychologists, Alan Leslie, Uta Frith and Simon Baron-Cohen. Autistic children do not seem to engage in social interaction that is typical in normal children of their age. They develop strange, repetitive behaviors, can become obsessed with details of particular objects and develop strange skills. Some of their development can be otherwise normal, as some have high IQs and many can talk. Yet they do not understand other people and often treat them like inert physical objects. Their special impairment is made obvious in simple tasks such as the "false-belief test" about puppets on a stage. Puppet 1 puts a marble in box A, then goes offstage. Puppet 2 arrives on the scene, finds the marble in box A, puts it in box B and goes offstage. Now puppet 1 comes back. The question is: Where will he look if he wants his marble? Children older than four years (and even Down's syndrome children of a comparable mental age) generally answer "in box A" and sometimes comment "in box A, because that where he thinks it is." Note that this apparently simple task is in fact quite complicated.
It requires that you hold two descriptions of the same scene in your mind: there is the actual location of the marble (in B) and the location of the marble as represented by puppet 1 (that's in A). These two descriptions are incompatible. One of them is false and is the one that will direct the puppet's behavior. Autistic children react like threeyear-olds. They expect puppet A to look for his marble in box B, because that is where it actually is. They do not seem to grasp that the marble in actuality could be in B and yet be represented by someone as being in A. Ever since these experiments were conducted in the 1970s many other experiments have shown that the autistic syndrome occurs as the outcome of an intuitive psychology deficit. For instance, normal five-year-olds, but not autistic children, assume that peering into a box will give them more information about what is inside than just touching its lid. Normal infants at the age of about ten months start pointing declarativelythat is, just to attract other people's attention to some object; they then check that people are indeed looking at the object of interest. Infants who do not do that often turn out a few years later to display the typical symptoms of autism.
Simon Baron-Cohen called autism a form of "mind-blindness", which is both felicitousautistic people are indeed impervious to something we think we "see", namely other poople's mental states and perhaps misleading. As Baron‑Cohen himself showed, our intuitive psychology is a bundle of different systems with different functions and neural implementations. One of these examines what other people's eyes look like and infers what they are looking at. Another one distinguishes things that move in an animate fashion from inert objects that move only when they are pushed or pulled. A third one computes the extent to which other agents perceive what we perceive, and in what way their perception is different. But autistic children seem impaired in only one special subcapacity: the representation of other people's representations. They are not blind to all mind-stuff but only to a most crucial part of it.
Representations of other people's actions and mental states may be even more complicated than this description suggests. For instance, studies of neural activation in normal subjects have shown that when we see someone making particular gestures, we generally imagine making the same gestures ourselves. Again, this is something we are generally not aware of. Studies show that the brain areas that are activated when we see people's gestures, overlap with those activated when we actually act in a similar manner. In other words, there is some part of the brain that is imagining performing the action witnessed, although it the plan is not made consciously and the motor sequence is inhibited. Infants seem to have the same system minus the final inhibition, which leads them to imitate other people's gestures. This may be the explanation for that famous and rather surprising experimental result, that people who merely watch others practice a particular sport actually get better at that sport (not quite as much as those who actually practiced, unfortunately). So seeing other people's actions or motions as something we might be doing is the job of yet another specialized system.
There is now evidence that representing other people's pain is the output of yet another specialized neural structure. Some sets of neurons respond selectively to excessive heat, cold and pressure. Other, neighboring areas respond selectively to similar events affecting otber people. The fact that we have specific emotional reactions to seeing pain inflicted on other people may result from the simple simulation produced by this system. That is, the experience of other people's pain, as handled by the brain's dedicated structures, to some extent overlaps with that of one's own pain. Again, there is a system that produces some description of how events affect other persons (in terms of a close simulation of what it would be like for us to experience the events) but that is concerned only with a narrow aspect of these events.
To sum up, then, our internal description of other people's mental life is not the product of a single, general theory of persons but the outcome of many different perceptions, simulations and inferences about different aspects of what they experience. What seemed a unified domain of "intuitive psychology" is in fact a collection of subdomains with specialized systems.
At several points, I have mentioned findings about infants and young children. The most striking and fascinating models of the mental encyclopediaor rather, of the systems making up what seems to be an encyclopediahave come from developmental psychology. This is not an accident. The study of young children asks the most radical of philosophical questionsWhere does knowledge come from? How do we ever manage to discover anything about the world around us?but turns them into scientific questions settled by experimental tests. We know a lot more now about how minds work because we have found out a lot about how young minds grow.
• Perception and understanding of surroundings require inferences and guesses about different aspects of objects around us.
• The mind is composed of specialized systems that produce inferences about these different aspects.
• Objects in different "ontological categories" activate different sets of these specialized systems.
• Each inference system is itself composed of even more specialized neural structures.