Jerome Feldman From Molecule to Metaphor
http://www.m2mbook.org/
Contents: From Molecule to Metaphor: A Neural Theory of Language
Part 1. Embodied Information Processing
1. The Mystery of Embodied Language
2. The Information Processing Perspective
3. Computational Models
Part 2. How the Brain Computes
4. Neurons and Other Cells
5. The Society of Neurons
6. Nature and Nurture
Part 3. How the Mind Computes
7. Connections in the Mind
8. Embodied Concepts and Words
9. The Computational Bridge
Part 4. Learning concrete words
10. First Words
11. Conceptual Schemas and Cultural Frames
12. Learning spatial relation words
Part 5. Learning words for actions
13. Embodied knowledge of actions
14. Learning action words
Part 6. Abstract and metaphorical words
15. Conceptual systems
16. Metaphors as mapping between (Cultural and Primary)
frames
17. Understanding as Simulation
Part 7. Understanding Stories
18. Aspect : from action to inference
19. Belief propagation in target domains
20. Understanding news stories
Part 8. Combining Form and Meaning
21. Combining Forms - Grammar
22. The Language Wars: Autonomy and Innateness
23. Combining Meanings - Embodied Construction Grammar
Part 9. Embodied Language
24.
Embodied Language Understanding
25. Learning Embodied Constructions
26. Remaining Mysteries
27. All Together Now
About the Author:
About
Professor Jerome A. Feldman, In His Own Words
People, including KQED's Michael Krasny who
has interviewed everyone, ask me where I get the chutzpah. My best answer is
that I was the first grandchild in the extended family and apparently was
always being told how smart and special I was. The neural connections that
constitute this belief have not been totally suppressed by the years of
experience to the contrary. This is hard on people who need to deal with me,
but it does seem prerequisite for anyone who would attempt to unify cognitive
science.
I don't know when I first became intrigued
by computers, the brain, and the mind, but it was well before my graduation
from Rochester in 1960. I had a student job with a professor of physiology and
vividly remember him arguing that the true path was through neuroscience.
But his research was testing the damage to the isolated crayfish giant axon
caused by different UV frequencies, funded by the Atomic Energy
Commission. Another defining undergraduate moment was when, in my physics
major, we reached areas of physics where intuition was (and is) not available.
My memory is of deciding that, if it was all math anyway, why not just switch
to a math major.
After one exploratory year in industry, I
joined the Math department of (then) Carnegie Tech. To my good fortune, the
department head was Alan Perlis, one of the pioneers of the field that would
become Computer Science. He said roughly: "you are bright and ambitious
but not smart enough to have a big impact on mathematics, but there is this new
field where nothing is yet known." I did a thesis with him on the
semantics of programming languages, but was more drawn to the AI effort at
Carnegie led by Allen Newell and Herb Simon.
The rest of the story is outlined in the
preface of the book. Cognitive Science literally involves the efforts of tens
of thousands of scientists. The NTL group's work over the years has been
primarily the product of the wonderful students that we are privileged to see
at Berkeley.
____________________________________________________________________________
Preface to "From Molecule to Metaphor"
I hear and I forget
I see and I remember
I do and I understand
Attributed to Confucius, 500
bce
Many years ago, I was browsing through
books on learning how to draw. One of them said, after a brief introduction,
put down this book and start drawing. This book is like that - it will
frequently suggest a simple mental exercise to help you personally experience a
phenomenon. If this appeals to you, you might like the book.
By now, virtually everyone agrees that
the scientific explanation for human language and cognition will be based on
our bodies, brains and experiences. The major exception is Noam Chomsky's,
whose dominance of 20th Century linguistics is unparalleled in any other
academic field. I will later quote from Chomsky's 1993 book, Language and
Thought , and the same idea was stated repeatedly in his 2003 Berkeley
lectures: We don't know nearly enough about the brain for cognitive science to
take it seriously. Chomsky has focused on linguistic form; since this
book deals first with meaning, we won't encounter him again until
Chapter 22.
As a first mental exercise, try expressing
to yourself what you know about how your own thoughts work. How do our brains
compute our minds? When I ask Berkeley students, on the first day of class, to
write a page on this question, most of the students express mystification. Even
among people who know a great deal about neuroscience, psychology, linguistics,
philosophy, and artificial intelligence there is often no clear idea of how the
findings of these fields could combine to yield even a preliminary
understanding of how language is embodied in us.
The purpose of this book is to propose the
skeleton of a theory that integrates current insights from many disciplines
into a coherent neural theory of language. It might seem that no such
effort is needed - isn't language obviously a function of our brains - what
else could it be? Certainly other human abilities such as motor control,
hearing, and especially vision have been studied as neural systems for many
decades. But language is still often treated as an abstract symbol system not
particularly tied to human brains or experience.
A great deal of permanent value has been
learned by formal studies of language, but it is surprising that the disembodied
notion of language persists. This is partly historical, but also arises from
the fact that other animals share our abilities in vision, etc. but not in
language. Much of the progress in neural theories of vision, motor control,
etc. have come from invasive animal experiments that are thankfully prohibited
on people. Until recently, there has been very little known about how language
is processed by our brains.
It is still true that no one currently
knows the details of how words or sentences are processed in the brain and
there is no known methodology for finding out. Many scientists believe
that it is premature (perhaps by centuries) to formulate explicit theories
linking language to neural computation. Even theoreticians are usually content
with suggestive models, which can't actually be right, but do suggest
interesting experiments. However, the cognitive sciences are revealing a great
deal about how our brains produce language and thought. And there is a long and
productive tradition, going back at least to the Greek atomic theories of
matter, of postulating "bridging theories" in advance of the detailed
evidence. Brian Greene's, The Elegant Universe, is a wonderful description of
the science of the fundamental nature of matter, where there might never be
experimental verification.
In contemporary science, it is not unusual
to have quite extensive knowledge at two ends of a causal chain and to build
and test theories that try to explicate the bridging links. For example,
astrophysics is concerned with linking fundamental particle physics with
astronomy. In economics and other social sciences, a principle concern is how
individual preferences give rise to group behavior. Similarly, much of
molecular biology is concerned with how genetic material yields the various
proteins and the resultant organisms. Higher levels of biology are also trying
to develop bridging theories. The search for a neural theory of language can be
seen as one of these attempts, albeit unusually ambitious. These bridging theories
are often developed as computer simulations and the book will follow this
tradition.
I treat the question of mind as a
biological one - language and thought are adaptations that extend abilities
that we share with other animals. For well over a century, this has been the
standard scientific approach to other mental capacities such as vision and
motor control. But language and thought, even now, are usually studied as
abstract formal systems that just happen to be implemented in our brains.
Instead, we will pursue the great ethologist Nico Tinbergen's (Tinbergen
1963) four questions that must be asked of any biological ability:
How does it work?
How does it improve fitness?
How does it develop and adapt?
How did it evolve?
The first three of these questions will be covered in
considerable detail. The origins of language are still largely unknown and will
be discussed briefly in Chapter 26.
There is a sufficiently large gap between
brain and language to contain ecological niches for many theories, especially
if their proponents are satisfied to ignore inconvenient findings. Understanding
language and thought requires combining findings from biology, computer
science, linguistics, and psychology. A theory that seems perfectly
adequate from one perspective may contradict what is known in another field.
Problems that seem intractable in one discipline might be quite approachable
from a different direction. Taking all the constraints seriously is the only way
to get it right.
But this requires us to understand the
essential ideas from several quite different scientific domains. In any of
these fields, keeping up with technical advances and doing original work is
extremely demanding and requires focused effort. There are some endeavors at
the boundaries between subfields, but very little scientific work that attempts
to encompass the full range needed for our task. I will need to synthesize a
bridging theory from separate fields, all of which have their focus elsewhere.
My approach is to pick out key findings and
theories from various disciplines and show how, in combination, they constrain
the possible bridging theories of language to a narrow family of possibilities.
Each discussion is an over-simplification
of some research field, often involving thousands of active investigators, and
thus is inherently incomplete. There are the usual references suggesting more
detailed discussions of various points, but these will be most useful as key
words for search engines. By the time you read this book, there will be
important new developments in each of these areas. A list of books for further
reading is included for people who would like additional background in one or
another direction.
While we are far from having a complete
neural theory of language, there have been enormous scientific advances in all
the relevant fields. Taken together, these developments provide a framework in
which everything that we know fits together nicely. The goal of this book is
simple; I would like you, at the end, to say: This all makes sense. It could
explain how people understand language. There will be no attempt to
convince you that other theories are wrong - in fact, I will assume that most
of them are partially right. The book can be seen as part of a general effort
to construct a Unified Cognitive Science that can lead the effort to understand
our brains and minds. I will try to present a story that is consistent with all
the existing scientific data and that also seems plausible to you as a
description of your own mind.
Except for one thing. There is one part of
our mental life that is still scientifically inexplicable - subjective
experience. Why do we experience everything in the way that we do? The pleasure
of beauty, the pain of disappointment, and even the awareness of being alive
... these do not feel to us like they are reducible to neural firings and
chemical reactions. Almost everyone believes that his or her own personal
experience has a quality that goes beyond what this book, and science in
general, can describe. If I had anything technical to say about
subjective experience, it would be the highlight of the book, to say the least.
People use terms like personal experience,
subjective experience, and phenomenology to label this idea. Philosophers have
coined a technical term, qualia, to refer to these phenomena that are currently
beyond scientific explanation. Antonio Damasio (Damasio 2003), who in my
opinion is doing the best scientific work on subjective experience,
distinguishes measurable emotions from subjective feelings. Aside from a brief
discussion in Chapter 26, this book will focus on what can be learned from
studying the physiological and behavioral correlates of experience - i.e., what
can be measured and modeled objectively.
My undertaking of this quixotic enterprise
came as the result of a year of explicit soul-searching around the time of my
fortieth birthday. I had the good luck of entering the field of Computer
Science in its infancy and believed that this gave me the opportunity to move
in almost any direction, exploiting insights into information processing not
available to previous generations. Having long-term interests in language and
the brain and having worked on various computer systems including some of the
earliest robots, I was led to focus on the question that I just asked you - How
does the brain compute the mind? Twenty-five years later, due to advances in
all fields that were inconceivable to me at the time, the outlines of an answer
seem to be emerging.
A Brief Guide to the Book
The book is designed to be read in order;
each chapter provides some of the underpinnings for later ideas. But it should
also be possible to look first at the parts that interest you most and then
decide how much effort you wish to exert. There are many forward and backward
pointers that may help integrate the material.
Information processing is the
organizing theme of the book. Language and thought are inherently about how
information is acquired, used, and transmitted. Chapter 1 lays out some of the
richness of language and its relation to experience. The central mechanism in
my approach to the Neural Language Problem is neural
computation. Chapters 2-3 provide a general
introduction to neural computation. Chapters 4-6 provide the minimal biological
background on neurons, neural circuits and how they develop. We focus on those properties
of molecules, cells, and brain circuits that determine the character of our
thinking and language.
Chapters 7 and 8 consider thought from
the external perspective and look at the brain/mind as
a behaving system. With all of this background,
Chapter 9 introduces the technical tools that will be used to model how various
components of language and thought are realized in the brain. A fair amount of
mechanism is required for my approach, which involves building computational
models that actually exhibit the required behavior while remaining consistent
with the findings from all disciplines. I refer to such systems as adequate computational models and believe that such
models are the only hope for scientifically linking brain and behavior. There
is no guarantee that an adequate model is correct, but any correct model must
be adequate in the sense defined above.
The specific demonstrations begin with a
study of how children learn their first words. This involves some general
review (Chapter 10) and a more thorough study of conceptual
structure (Chapter 11) that is needed for word learning. The first
detailed model is presented in Chapter 12, which describes Terry Regier's
program that learns words for spatial relation concepts across languages. This theme of concrete word learning is then
extended to cover words for simple actions in Chapters 13 and 14, which describes David Bailey's
demonstration system.
The next section extends the discussion to words for abstract and metaphorical concepts. In Chapters 15 and 16, we look further at the structure of conceptual systems and how they
arise through metaphorical mappings from direct experience. Chapter 17 takes
the informal idea of understanding as imaginative
simulation and shows how it can be made the basis for
a concrete theory. This theory is shown in Chapter 18 to be sufficiently rich
to describe linguistic aspect - the shape of events. This is enough to capture
the direct effects of hearing a sentence, but for the indirect consequences, we
need one more computational abstraction of neural activity - belief networks, described in Chapter 19. All
of these ideas are brought together in Srinivas Narayanan's program for
understanding news stories, discussed in Chapter 20.
Chapters 21 - 25 are about language
form i.e., grammar - how grammar is learned and how grammatical processing works.
Chapter 21 lays out the basic facts about the form of language that any theory
must explain. Chapter 22 is partly a digression; it discusses the hot-button
issues surrounding how much of human grammar is innate. We see that classical
questions become much different in an explicitly embodied neural theory of
language and that such theories can be expressed in standard formalisms
(Chapter 23).
Chapter 24 shows how the formalized version of neural grammar can be
used scientifically and to build software systems for understanding natural
language. The poster child for the entire theory is Nancy Chang's program
(Chapter 25) that models how children learn their early grammar - as explicit
mappings (constructions) relating linguistic form to meaning. Chapter 26
discusses two questions that are not currently answerable: the evolution of language and the nature of
subjective experience. Finally, Chapter 27 summarizes the book, and suggests
that further progress will require a broadly based Unified Cognitive Science.
But the scientific progress to date does support a range of practical and
intellectual applications and should allow us to understand ourselves a bit
better.
A version of the material in this book has
been taught to hundreds of undergraduate students at UC Berkeley over the
years. There were weekly assignments and most of the students actually did
them. The course did not work for all the students, but a significant number of
them came out of the class with the basic insights of a neural theory of
language. If you want to understand how our brains create thought and language,
there is a fair chance that the book can help.
__________________________________________________________________________________
The 
(NTL)
project is an interdisciplinary research effort to answer the question: How does the brain compute the mind?
http://www.icsi.berkeley.edu/NTL/
From Molecule to Metaphor:
A Neural Theory of Language
From Molecule to
Metaphor: A Reader's Roadmap
This Reader's Roadmap has a one-page diagram for each of the 27 chapters of
the book, highlighting key ideas and their relation to (mostly) earlier
chapters. Believe it or not, the first version was done by John Torous, a student in my CS182 class, as a way to study for the final exam. This
slightly cleaned up version was assembled with the help of Leah Hitchcock and
Jacob Wolkenhauer.
http://www.m2mbook.org/reader-roadmap/
http://www.m2mbook.org/reader-roadmap/CH27.jpg
