Dunbar - The Human Story
Here is a very simple test you can do with any child. Sally and Ann are two dolls. Sally has a ball. She puts the ball under the cushion on the chair. Then, she leaves the room. While she is out of the room, Ann takes the ball out from under the cushion and hides it in the toy box on the other side of the room. Later, Sally comes back into the room. Where does Sally think her ball is?
Up to the age of about four years, a child will instinctively say: 'Sally thinks the ball is in the toy box.' A child of this age cannot distinguish between its own knowledge of the world and that of other individuals. But between the ages of four and four and a half years, the child passes through a rapid phase of understanding. From about four and a half onwards, it will answer the question by saying that Sally thinks her ball is under the cushion,'. . . but I know it's not.'At that point, the child is able to recognise that another individual can have a belief about the world that is different from its own, a belief that it knows (or at least thinks) is untrue. At this point, a child is said to have acquired a 'theory of mind' - it instinctively understands that others have minds of their own not unlike what it experiences as its own mind. This form of understanding, sometimes also known as 'mind-reading' or 'mentalising' is a remarkable and crucial feature of human psychology.
Tests like the Sally-Ann test are called 'false belief' tasks because to pass them the child needs to understand that another individual can hold a false belief (one the child knows to be untrue, or at least one that is different from the one the child supposes to be true). There are now a number of these tests. Another is called the 'Smartie Test' In this test, you show the child a tube of Smarties and ask: What do you think is in the tube? The inevitable answer is: Smarties. You take the cap off and the child sees that the tube actually contains pencils. After replacing the cap, you then say to the child: 'I'm just going to bring your friend Jim into the room. What do you think Jim will say is in the tube?' Up to the age of four, children will invariably say'pencils'because they cannot distinguish between their own knowledge of the situation and someone else's; but after about four and a half, they will reply, with rapidly increasing conviction,'Smarties.'
Tests of this kind have become benchmarks of children's developing abilities for inferring the mental states of others. They represent a critical Rubicon in the process of child development because they demarcate the moment at which children can begin to engage with an imaginary world that is not physically present. They can now begin to engage in those forms of pretend play that are so characteristic of younger children - to imagine that the doll really is alive and can suck liquid from the end of a rag or from a baby bottle. They can take part in dolls' tea parties, pretending that the empty teapot really does contain tea to pour into the cups, and afterwards that they can themselves drink 'real' tea from the patently empty cups.
Herein lies the great mystery of child development, for children are not born with this ability. Infants and toddlers alike treat the world as being exactly as they experience it. They cannot imagine that it could be other than what they perceive it to be. They lack the ability to imagine. And because they cannot imagine that the world is other than what they know it to be, they cannot suppose that another individual - child or adult - believes something to be the case that they know is not. And as a result, they cannot do something that is in some ways the hallmark of the grown-up world: they cannot exploit another individual's view of the world to feed them a lie.‑
The Art of Mind-reading
At this point, I need to introduce a technical term. Some decades ago, philosophers interested in the nature of minds coined the term 'intentionality' to refer to the kinds of mental states that we have when we are conscious of holding some kind of belief, desire or intention. The term refers collectively to mind-states like knowing, believing, thinking, wanting, desiring, hoping, intending, etc. It refers to the state of being aware of the contents of your own mind. Intentionality can be conceived of as a hierarchically organised series of belief-states.
In this scheme of things, computers are zero-order intentional entities: they are not aware of the contents of their 'minds' Some living organisms such as bacteria (and perhaps some insects) may also be zero-order intentional beings.
Most organisms that have brains of some kind are probably aware of the contents of their minds: they'know' that they are hungry or 'believe' that there is a predator under that bush over there. Such organisms are said to possess first-order intentionality. Having a belief about someone else's beliefs (or intentions) constitutes second order intentionality, the criterion for theory of mind (or, as it is more often known in the technical literature, ToM). Jane believes that Sally thinks her ball is under the cushion. Jane has two belief states in mind (her own and Sally's), so theory of mind is equivalent to second-order intentionality.
We humans can dearly go beyond this level, however. Peter wants Jane to suppose that Sally thinks that her ball is still under the cushion. Sally is in first-order intentionality, Jane in second and Peter in third. It seems that there is an upper limit on what we can do in this respect. Conventional wisdom suggests that adult humans have an absolute upper limit on the levels of intentionality that they can cope with at about five or six orders: Peter believes  that Jane thinks  that Sally wants  Peter to suppose  that Jane intends  Sally to believe  that her ball is under the cushion.
If your mind has just been turned inside-out by this sentence, it is not too surprising: few adult humans can keep who's thinking what here straight because, in the limit, there are too many orders of intentionality (marked out by the numbered verbs in italics) for us to keep track of. Most everyday situations probably require no more than second-order intentionality and, in actual practice, the limit for most people is probably about fourth or fifth order: Jane thinks  that Sally wants  Peter to suppose  that Jane intends  Sally to think  that [something concrete is the case]. We've just lost Peter believing.
We know this is where the limit is because we ran some tests to see just how well normal adult humans do. At the time we ran the tests, no one really knew the limits to human abilities in this domain. All the work had focused on theory of mind (second-order intentionality) and most of it had been carried out on young children around the transition point where they acquire ToM. In order to explore the capabilities of normal adults, we devised some ToM stories that went up to six orders of intentionality.
I would like to have been able to say that we set sixth-order intentionality as the limit on these stories for some incredibly sophisticated scientific reason. Alas, in reality, the truth is that I found it impossible to write a convincing seventh-order ToM story. The stories were short vignettes of everyday life in about zoo words: someone wanting to make a date with a girl that he thought fancied someone else, or someone wanting to persuade her boss to give her a pay rise by pretending that she had been offered another job elsewhere. Anything longer than sixth order proved so tortuous that I ended up getting completely confused myself. A bottle of whisky later, and well into the early hours of the morning, I gave up and settled for sixthorder stories.
We gave the tests to about 120 university students. They were read the stories and were then asked to answer a series of questions about who was thinking what. Between 80 and 90 per cent of the subjects answered the questions correctly at any given level up to fifth-order intentionality. That seemed very reasonable. But the subjects'performance went decidedly pear-shaped with sixth-order questions: only about 40 per cent of them got these right. This is a dramatic and sudden collapse in what had, up to that point, been a very competent performance by a large sample of young adults with aboveaverage IQs. These results seem to be quite robust, since we were able to confirm them in a second study carried out some time later by Jamie Stiller using stories that went up to ninth order.
We know this sudden collapse at around fifth order is not just a memory problem, because we also asked them questions about the factual content of the stories in between the various ToM questions, and they had no problem at all with these. They remembered the main factual events in each story. We also gave them a simple factual story detailing a sequence of causally related events about an old man setting fire to himself when he fell asleep smoking. This story shared the same kind of embedded hierarchical structure as the ToM stories (A causes B, which causes C, etc). Subjects had no problems with this story: the percentage of correct responses remained constant at about 90-95 per cent right up to seventh-order concatenations of causal sequences ('When A happened, B followed, which set C off, which resulted in D, which triggered event E, which gave rise to F, which precipitated G'). So the problem seems to be something to do with the reflexivity of mental states rather than just causal sequences per se.