EFTA01092999.pdf

1 The Computational Structure of Mental Representation Joscha Bach Berlin, February 26th, 2013 Now is the time for starting a new Artificial Intelligence initiative. What is the mind? Last month, a grand proposal by Barack Obama made headlines everywhere in the world: He suggested a large-scale initiative -this is arguably the most to create a detailed map of the activity of the human brain, at interesting question that the level of individual neurons. This idea is quite similar to (and likely inspired by) Henry Markram's Blue Brain project in our species can ask itself. Lausanne, which recently won a 1.5Bn grant from the European Union. While this is an interesting project in its own right, it will not address the key question of cognitive science: What is the mind? What are the building blocks and fundamental operations of thinking and perception? Here, I would like to suggest the instigation of a project that is at once more ambitious, more narrowly targeted, and likely to yield more profound theoretical, practical and cultural insights than the Brain Activity Mapping initiative: The study of the computational structure of mental representation. Mental representation, not Cognition is incidentally enabled by human nervous systems, but in its nature, it is not a chemical, biological or physiological neurons will inform the phenomenon. Instead, cognitive systems are a class of core our understanding of information processing systems, thinking is a set of certain, functionally identifiable operations, over certain, functionally cognition. identifiable types of representations, and with respect to a set of problems and properties given by certain environments. Thus, the Structure ofMental Representation initiative should focus on mental content, with all its dynamic, relational, conceptual, and linguistic elements, and it's grounding in perception and interaction. EFTA01092999  2 The study of mental Unraveling mental representations will be an interdisciplinary effort among several cognitive sciences, but distinct from the representations requires a disciplines that we find now: new scientific discipline The project will differ from linguistics, for instance, in similar way as geography differs from plate tectonics. Where linguists study the intrinsic 'geography' of languages, and the structural commonalities among them, mental representations uncover the underlying dynamics that produce natural languages (as the solutions to the problem of translating between hierarchical, distributed, associative, ambiguous representations and the discrete strings of symbols that we use as a means of exchanging and organizing ideas). Integrating concepts from Mental representations lie outside the domain of neuroscience, which mostly focuses on material descriptions of the function linguistics, cognitive of the underlying substrate. They cannot be studied well within psychology and contemporary psychology, which favors an experimentalist, quantitative approach, where we need to address qualitative neuroscience within the questions by constructing working, implementable systems. methodological framework And needless to say, the study of mental representations is currently not well represented in Artificial Intelligence of constructionist Artificial research, which does provide a productive methodology, but Intelligence has mostly turned towards applications and narrow Al solutions. Despite the lack of a common methodological ground between the cognitive sciences, we can now observe a growing consensus on how to approach the problem of modeling the mind, and its representational apparatus. During the last decade, a number of initiatives have sprung up within Al, psychology and cognitive science, each with journals, A new common ground workshops and conference series, and a large personal overlap between Al and cognitive among each community. Examples include Artificial General Intelligence (AGI), Biologically Inspired Cognitive Architectures science researchers (RICA), Cognitive Systems, and Cognitive Modeling (ICCM). Among those groups, there is an emerging consensus on several paradigms: Integrated architectures of cognition. We need to study whole, working systems, both for the purpose of comparison between approaches, but mainly, because cognition is not the product of the activity of individual modular functions, but of the interaction between them. Universal representations. Representations must provide both distributed and localist aspects, to enable neural learning, information retrieval via spreading activation, as well as symbolic processing (language and planning). Perceptual grounding. Representations should be grounded in an environmental interaction context, using a bottom-up/top-down perception paradigm, to allow for EFTA01093000  3 open-ended autonomous learning and language acquisition. Integration of perception and action. Environmental interaction is (at least partially) a control process that requires a modeling of the relationship between sensory data and operations performed by the system. (Semi-) Universal problem solving, enabling paradigms of learning, planning, reasoning, analogies, language acquisition and reflection. A structured memory, including provisions for a world/situation model, a protocol of environmental interactions, procedural memory, declarative and typological abstractions, a model of self, and a 'mental stage', to facilitate anticipation and planning. Decision making and motivational mechanisms, to address both autonomous, goal-directed cognition in complex, open domains, and the genesis of goals and intentions. The direction and modulation of attention, and the emergence of emotion and affect, which are configurational aspects of cognition that either reflect the allocation of cognitive resources, or structure social interaction. Constraints for Mental Representations Based on the emergent paradigm of a new generation of cognitive architectures, we can think about mental representations in a new and productive way. More specifically, we know that mental representations include perceptual and propositional/conceptual content and we are aware of many constraints. Mental representations must Offer support for both connectionist and symbolic processing (including compositionality and grammatical structures), with the latter one being a special case of the former Possess a hierarchical structure, bottoming out in sensory perception Cover prototypes, Individuals and abstractions Include perceptual and relational features, objects, situations, and episodic knowledge Allow simulation of dynamic processes Solve the bridge problem between fuzzy associative hierarchies and the discrete symbol strings of natural language EFTA01093001  4 Operations on these representations include reinforcement learning and classification, perceptual top-down/bottom-up processing, abstraction, analogy formation, associative and syllogistic reasoning, planning, reflection, anticipation and several modes ofreorganization. Creating a scientific community The task of the Structure ofMental Representation initiative will at first consist in the creation of a large and competitive community, built around a set of benchmark problems. I suggest picking a set of problems that covers most of the above architectural requirements: the comprehension of dynamic visual sequences (movies) and narratives. The evaluation of comprehension may focus both on a discourse level (asking the system general and specific questions about the consumed visuals or narratives) and directly, by producing dynamic renderings and depictions of knowledge represented within the system. At least on the discourse level, a direct comparison to the functional properties of child performance at different cognitive stages is possible. The Comprehension Turning the benchmark task into a regular competition allows a direct comparison between models, and offers a strong Challenge incentive for exchange of solutions among research groups. The need to for broad solutions with given material and intellectual resources will enforce a higher degree of the reuse of code and ideas than we currently see in Al architectures (outside of robotic soccer, where such competitions have turned out to be highly successful). By structuring the comprehension tasks into different levels and sub-domains (such as basic language acquisition tasks, grammar, perceptual and logical tasks, social reasoning/theory of mind, affective evaluation, constructive problem solving, use of analogies and metaphor etc.), we can formulate consistent long-term goals and realistic short-term problems. The Formation of a Pilot Team To pull the Structure ofMental Representation initiative off the ground, we will need to create a pilot team, which is the object of this proposal. The pilot team has the following tasks: - Bring a critically sized group of people together, with a well-defined set of shared goals - Formulate an initial set of hypotheses Bring these in the context of an existing or hypothetical cognitive architecture, so specifications can be derived EFTA01093002  5 Build an initial software infrastructure for testing benchmark problems, and to create a common methodological ground Composition of a pilot The pilot team should consist of a scientific board, including a mutually compatible sub-set of contemporary thinkers in the team field. Its core should consist of a new generation of researchers that have the resources and impetus to work full-time on the design, implementation and publication of the project's infrastructure. The initial team size should not be large, but agile and effective (I would suggest about five permanent researchers, about the same number of dedicated software developers, and 2-3 slots for visiting scientists at a time). The initial duration should be about 2.S years, followed be an evaluation, and a possible continuation of similar length. Possible candidates for the scientific board include (in no particular order) Paul Rosenbloom (who worked with Alan Newell on the cognitive architecture Soar, and now started a new architecture), John Sowa (well-known for his groundbreaking work in semantic networks), Ben Goertzel (designer of the cognitive architecture OpenCog), Luc Steels (an Al researcher studying the emergence of natural language in communities of software agents), Jerome Feldman (who devised building blocks for a Neural Theory ofLanguage), Stephen Pinker (who thought much about the relationship between language and mental representation), Stephen Kosslyn (an expert on the psychology of mental representation), Daniel Dennet, Cristiano Castelfranchi (an Al researcher specializing on mental representations of intentions and social context), and many others. The project should start out with a number of kickoff meetings to invite the interaction between suitable candidates, and the formation of a core group. The eventual pilot team could be located in many possible places, however, the project should be able to draw on the vicinity of academic and other infrastructure, and the influence of a creative environment About me (Joscha Bach): Driven by an interest to learn hot the mind works, I studied philosophy and psychology, graduated in computer science/artificial intelligence, lead several small academic research groups, built the cognitive architecture MicroPsi and obtained a PhD in cognitive science. I co-founded a couple of startup companies in Berlin, but turned back towards academia to contribute to the question of how to build a mind. EFTA01093003