BROCA: A Computerized Environment for Mediating Scientific Reasoning through Writing

Abstract: This paper describes a work-in-progress: a computerized learning environment for teaching the conceptual patterns of scientific reasoning. BROCA (Basic Research, Observations, Critical Analysis) is theory-driven, combining two very powerful conceptual models of thinking. The first -- drawn from cognitive psychology and information theory -- focuses on the mental manipulations by which data becomes information and information becomes knowledge. The second theoretical construct comes from rhetoric and describes the intellectual activities carried out in prewriting, drafting, and revision by an expert writing. As an interactive "cognitive tool," BROCA provides scaffolding (through visual algorithms and adaptive prompting) to help a fledgling thinker practice the robust patterns of scientific reasoning.

Keywords: Computers and Education; Computer Uses in Education

Category: K.3, K.3.1

1 Introduction

The written word is crucial to science for at least two compelling reasonings. First, the texts of science -- publications that report findings -- are the life blood of progress for a community of researchers. Framing a question of interest, designing an experiment, collecting data and observations, analyzing the raw findings, and interpreting results in light of theory -- all these demanding tasks are the process of science, but the written work provides the vehicle for dissemination of the product.

Second, writing is important to science because the act of placing ideas into language mediates higher-order intellectual activities that are foundational to scientific thinking [Kuhn, Amsel, & O'Loughlin, 1988]. Though other symbol systems play a major role in scientific reasoning, language fosters mental

manipulations such as synthesis, analysis, classification, inferencing, definition, hierarchical order, comparison/contrast, elaboration/extension. Some of the most respected of twentieth-century educational theorists have endorsed this notion of writing as a heuristic for learning and for understanding. Vygotsky [Vygotsky, 1962], Luria [Luria & Yudovich, 1971], and Bruner [Bruner, 1971], to name only a few, have pointed out that higher cognitive functions seem to develop most fully only with the support of verbal language -- particularly, of written language.

In broad terms, the process of scientific inquiry mirrors the writing process. Both the scientist and the writer go through an initial gathering and sorting out of ideas, from which tenuous though testable explanatory notions are made (hypotheses for the scientist and thesis statements for the writer). After considerable trials, sound relationships between entities are found (experimentation for the scientist and drafting for the writer) and a supportable belief structure emerges. Continued scrutiny (replication in science and revision in writing) results in knowledge -- an artifact that can take its place in the body of received opinion. Because of this similarity of intellectual activities, writing can be used as an analog for scientific thinking.

Much of today's educational reform in the United States focuses on declining competency in mathematics and science. Of particular concern is the drop in abilities labeled "scientific reasoning" and "scientific literacy." A national consensus has emerged that education in these subjects must be renewed and improved. In particular, advocates for change call for innovations that:

• Incorporate findings from basic cognitive research to build a "modern" pedagogy.
• Integrate the various performative skills foundational to science, rather than isolating them in artificial subsets.
• Teach the process as much as the product of science.
• Include advanced computer-mediated educational technologies for instruction.
• Accommodate individual differences in learning styles, gender, and background.
• Make the concepts of science accessible through a naturalistic curriculum and authentic exercises, guided-inductive learning, and cognitive apprenticeship models of instruction.

We propose a multimedia, interactive learning environment that weds forms of scientific inquiry to forms of scientific discourse. BROCA (Basic Research, Observation, and Critical Analysis) is an end-to-end knowledge development environment that mediates the intellectual activities implicit in scientific thinking. Our claims are that BROCA will:

• Increase competence, creativity, and confidence in formal reasoning.
• Promote and sustain interest in scientific investigation.
  Page 572

  ---

• Engage the user and transfer powerful strategies for problem-solving.
• Bridge the gap between process (thinking) and product (writing).
• Improve quality of writing by improving quality of thinking.

As currently conceived, the system is intended for professionals-in-training in disciplines concerned with brain research and clinical outcomes. An immediate market consists of colleges, universities, and research laboratories/institutes where BROCA could be fielded as stand-alone training technology. With some reengineering, an "empty engine" version of BROCA could be marketed as a productivity tool to all segments where knowledgeworkers require concept processors as replacements for word processors.

2 Background and Technical Approach

Until recently, philosophers provided much of the explanatory theory for what constitutes scientific thinking. Within the past few decades, however, increasing research by both psychologists and science educators has given us a more distinct working model of the processes and mechanisms by which scientific reasoning take place. Inhelder and Piaget [Inhelder & Piaget, 1958] characterize scientific thinking as "combinatorial cognition." In other words, scientific thinking consists of second order mentations, or "operations on operations" in the sense that the scientist uses reflection (second order mentation) to extract additional meaning from the products of first order mentation such as classification and relation. Newell and Simon [Newell & Simon, 1972] propose that scientific thinking be characterized as a process of search through a problem space, whose complexity is managed by the setting of goals and by invoking a collection of heuristics for partitioning the task into sub- "spaces" which can then be examined through powerful and productive strategies. Holland, Holyoak, Nisbett, and Thagard [Holland, Holyoak, Nisbett, & Thagard, 1986] explain the making of meaning in science as a process of induction. The model these authors describe is essentially a production system made up of condition-action rules. They account for contextuality in their model of inductive reasoning as follows: associations, patterns, regularities are observed, and on the basis of expectations or concepts about organization in the domain, a new, potentially superordinate concept is formed. Additionally, these researchers admit the importance of self-consciousness or metacognition by including a discussion of the "model of the model." Such robust theories have been extended into a new pedagogy which insists that science educators ought to be teaching methods of scientific thinking rather than merely scientific knowledge or concepts [Lawson, 1983].

2.1 Foundations in Theory

We base the design of BROCA on two explanatory models: one from information science and one from cognitive science's finding on discourse strategies. We do not intend our resultant hybrid model to be reductive or prescriptive. Rather, we see this "cognitive tool" [Salomon, 1993], [Salomon, 1988], [Salomon, Perkins, & Globerson, 1991] as an enabling prosthetic to help scientists-in-training recognize and take control of demanding intellectual processes.

Our first model comes from the general premise that there is a qualitative difference between data, information, and knowledge. Data are discrete entities, such as facts. While a body of data assembled for a specific purpose undoubtedly has relationships among the elements, these patterns of meaning must be extracted through an examination of the set and a filtering out of extraneous pieces. The systematic processing of data produces information, or statements about the associations in the data. Information is a codification and a clarification of the connections in data that produces more compact and easily remembered interpretative statements. A synthesis of information builds belief structures more comprehensive and more resilient than those contained in information.

Thus, knowledge is more powerful than information in that it has predictive or explanatory value. In other words, knowledge structures become both the product and the continuing impetus for such intellectual activities as exercising judgment and making rational decisions. These differences -- and more importantly, the transformations that take place when moving up the ladder of abstraction -- are the foundations of human memory and thinking. Moreover, this hierarchical model of knowledge evolution is central to many of the emerging applied fields coming out of the cognitive sciences.

Our second model comes from the cognitive science research into the importance of strategy acquisition for thinking and writing. Generalized models of good thinking include an array of techniques (strategies) for accomplishing goals, knowledge about when and how these techniques should be used (metacognition), and an extensive, task-specific knowledge base that is used in conjunction with the strategic and metacognitive processes [McCormick, Miller, & Pressley, 1989]. Realization that good thinkers have a repertoire of problem-solving behaviors for various types of tasks launched a new pedagogy for strategy acquisition.

Strategies are powerful manipulations by which the problem-solver (1) defines the task and makes analogs to other similar situations, (2) prunes away extraneous elements or eliminates "noise" from the problem space, (3) mediates state transformations, such as clustering specifics and making superordinate categories, and (4) links new knowledge with prior knowledge.

Figure 1 illustrates how we have conjoined the two models and used the process of scientific reasoning merged with the strategies of discourse to create a cognitive architecture by which BROCA melds scientific reasoning with the rhetoric of science. Basically, we specify the problem spaces (data, information, knowledge) as

arenas for exercising scientific thinking and the state transitions as discourse manipulations necessary to create a tangible, codified "belief structure" which will be passed up the ladder of abstraction as a value-added artifact to be operated upon in the next higher-level workspace.

We have modeled the process as three problem-spaces (data, information, knowledge) and three state transitions (prewriting, writing, revision). Manipulations within each of the problem-spaces requires its own specific set of heuristics. For example, within the data workspace, the thinker is basically examining evidence against a set of goals or expectations. This guided exploration involves a set of procedures or rules-of-thumb by which the thinker sorts out the noise from the data.

Methods guiding the exploration are an established set of principles that the thinker has learned, either formally or informally.

2.2 Extending Theory into Instructional Design

Extending a psychological theory into an effective pedagogical enactment usually proves difficult. Though models help to identify the components of thinking, the process is not linear and teaching is not as simple as providing instruction for each component and then putting the parts together. At the heart of the matter seems to be a kind of paradox. Accomplished problem-solvers in any domain demonstrate considerable finesse in higher-order intellectual activities: critical thinking as a precursor to knowledge; judgment as the foundation for decision making; and self-confidence as the initiator of self-monitoring. Unfortunately, novice problem-solvers are so overwhelmed by the heavy demands of the process that they seldom are able to exhibit -- even on a rudimentary level -- these higher-order activities. Thus, they are precluded from working with the very same cognition that would make them better at their task.

How can powerful strategies be taught without divorcing them from the context in which they naturally appear? David N. Perkins [Perkins, 1986] extracts the notion of the "thinking frames" from the more abstract cognitive construct of schema-driven strategy development [Norman, Gentner, & Stevens, 1976]. Perkins offers the following definition:

...[A] representation intended to guide the process of thought, supporting, organizing, and catalyzing that process. This representation may be verbal, imagistic, even kinesthetic. When well-practiced, it need not be conscious. A thinking frame, in order to organize our thinking, includes information not only about how to proceed but when to proceed in that way (p. 7, italics in the original).

In practice, thinking frames occur in a number of different domains. Their form spans a gamut from simple (but powerful) mnemonic devices for extending the working memory to rich mental models that foster expert behaviors by invoking appropriate strategies, conserving and allocating mental energies, and orchestrating steps in staged problem-solving techniques. In the realm of problem representation, John Hayes [Hayes, 1981] notes the utility of simple visualizations (such as matrices) to delimit the problem space and to facilitate "search" in a complex situation. For example, Jones, Amiran, and Katims [Jones, Amiran, & Katims, 1985] found that using a grid to encourage name-and-attribute clustering aided in recall and systematically produced effective compare-and-contrast type essays in a study of young adults. The rich body of research into mental models and instructional design [Gentner & Stevens, 1983] [Kieras, 1988] suggests that highly-complex, multi-dimensional intellectual activities can be represented so as to help the novice activate and amplify specific expert strategies. For example, Gentner and Gentner found that the types of thinking people could do about electrical circuits depended on what kind

of analogy (teeming crowds or flowing water) was used to represent the system and -- consequently -- the inferences these metaphors enabled [Gentner & Gentner, 1983].

Research indicates that teaching strategies isolated from their context does not produce enduring results [Garner, 1987]. What is needed is an instructional system that serves as a procedural facilitator. This term is used by Vygotsky [Vygotsky, 1962] to explain the cognitive mentoring and developmental dynamics that occur between master and apprentice and between peers during collaboration. Salomon, Globerson, and Guterman [Salomon, Globerson, & Guterman, 1989] and Zellermayer, Salomon, Globerson, Givon [Zellermayer, Salomon, Globerson, & Givon, 1991] use the term to suggest that computer technology can serve as a partner for the fledgling student and provide the scaffolding that allows the novice to practice the more robust problem-solving behaviors of an expert. In brief, such a procedural facilitator for scientific reasoning would:

• Ease demands on short-term memory and help to focus attention on strategically important aspects of the task.
• Guide the inculcation and self-initiation of higher-order processes (metacognition) which the novice is unlikely to activate without prompting.
• Explicitly model strategic intellectual processes so that the learner avoids what Collins & Gentner [Collins & Gentner, 1980] have termed "downsliding," or becoming increasingly entangled in lower and lower levels of mental actions, finally concentrating all attention on surface features and trivial aspects of the task to the exclusion of larger concerns in the process.
• Mediate transitions from abstract thoughts to symbolic representations. For example, in the domain of writing, Smith & Lansman [Smith & Lansman, 1989] conceptualize composition as an activity that takes place in three modes: thinking, organizing, and adapting. Each mode has its own set of goals, processes, and constraints. What is needed, according to these researchers, is a set of visual workspaces that help the novice writer to move gracefully through the various state-transitions inherent in moving from thoughts to finished text.
• Provide embedded strategic models for higher-order cognitive activities (such as discerning patterns in bodies of information, decision-making, staged problem solving, analysis, synthesis, and inferencing).

Pea [Pea, 1985], Perkins [Perkins, 1985], and Salomon, et al. [Salomon, 1993], [Salomon, Perkins, & Globerson, 1991], [Salomon, 1988] make a compelling case that some types of computer applications not only facilitate a task's accomplishment but also help the user to internalize profound strategies for later performance of the same or similar tasks.

Pea [Pea, 1985] makes a distinction between "amplification" and "reorganization" in examining the effects of cognitive technologies. Defining "cognitive technology" as "... any medium that helps transcend the limitations of the mind, such as memory, in activities of thinking, learning, and problem solving." (p. 168), Pea notes

that a child can extend her short-term memory with paper and pencil by writing down a long list of words. The short-term memory is amplified in this single instance, but the child's mental capacity has not been improved or altered -- unless the paper and pencil somehow prompt the child to "chunk" the words in more easily processed clusters. In short, cognitive tools that have been carefully designed to move beyond mere conveniences teach strategies for mental activities and -- rather than deskilling -- leave their users better off for having engaged the tool.

Pea uses the electronic spreadsheet as "... an illustration of computer technologies that can reorganize, and not merely amplify, mental functioning...." (p. 170); therefore, examining the cognitive design features provides us with a point of departure. First, this electronic representation recaptures all the features and functionality of the paper ledger sheet. A two-dimensional array displays categories and attributes, creating individual cells for the placement of data. Even this static ordering enables various forms of complex intellectual activity -- such as inferencing and categorical reasoning.

Placing this representation in electronic form adds new dimensions: now the user can dynamically manipulate data in each cell and watch the impact of change on other elements of the system. This, contends Pea, changes the level of engagement between the material and the user. Instead of merely entering data, the user is empowered to perform financial modeling, forecasting, and other forms of systemic thinking. Recent empirical studies support Pea's claims that using the tool elevates the user's understanding of domain constructs in ways that both endure and generalize, even without the presence of the tool.

Our second example, the abacus, demonstrates how a non-western, non-computerized device can mediate cognitive reorganization. Miller and Stigler [Miller & Stigler, 1991] studied abacus use as a example of research questions inherent in a whole category of representational systems. The abacus has utility in that one uses it to complete specific, well-defined mathematical manipulations. However, in mastering the device, one also gains insights into the conceptual dimensions of the domain. In one study, students who had reached a level of proficiency with the abacus are also better able to answer sophisticated questions about number theory than a set of matched controls who had no exposure to the abacus. Within a wider context, the abacus as cognition reorganizer could lead to questions of minimalism: How small and simple can a device be and still leave a "cognitive residue" by reorganizing mental abilities? Abacus studies show that even within the non-computer examples, cases exist of a simple but eloquent teaching device not only having a pragmatic result but also a conceptual effect.

As a cognitive tool, BROCA encourages its users to move from exploration to final report in a multimedia environment where scaffolding and visual algorithms gently guide the thinker through multi- staged intellectual activities. The software encourages the user to engage in powerful strategies that foster guided-inductive thinking and ensure mindful engagement in the task.

2.3 Extending Instructional Design into Software

The rhetoric task modeled by BROCA involves a type of writing familiar to managers, investigative reporters, engineers, and researchers. In issue-based writing, source inputs (text, graphics, numbers) become the raw materials in composing a position paper, an evaluative summary, an interpretive response. BROCA is a comprehensive, integrated environment providing cognitive support for professionals or professionals-in-training who create complex documents as part of their job. It differs from many other computer-aided writing tools in that it is an end-to-end development tool. It assists the writer throughout the process, from generation of ideas to production of connected prose.

As indicated by Figure 2, BROCA uses Bereiter & Scardamalia's [Bereiter & Scardamalia, 1987] notion of a dual problem-space model for writing: a content space (essentially, summarizing, analyzing, and synthesizing information about the topic) and a rhetoric space (essentially, planning and organizing the domain information into a logically and stylistically appropriate formal text artifact). Six distinct cognition enhancers work in tandem to mediate the multi-staged process.

Figure 2: Six Cognition Facilitators of BROCA

The tools vary in their nature and can be roughly classified by placing them on a continuum showing how they mediate the knowledge development process. In general, the six tools operate in a fashion analogous to a database management

system (DBMS). Each of the six environments permits the user to call up materials from the object-oriented, multimedia database and manipulate, extend, or connect entities in response to the adaptive instruction and cognition facilitators embedded in the workspace. The movement is sequential through the six tools; thus, gains made in one environment are consolidated and enriched in the next workspace. The following segments briefly describe the types of intellectual activities and the visual representations contained in each tool.

Cluster Browser: Based on a hypertext platform, this tool encourages the early exploration of a body of data. The user examines the evidence and constructs preliminary evaluations or carries out additional analysis and calculation. Thinking frames and adaptive instruction mediate two powerful categories of intellectual activities: (1) transforming data structures (converting verbal structures to numerical; numerical to graphic, and the like) and (2) working with semantic networks (constructing a rudimentary set of relationships among the various data in the workspace).

The resultant "belief structure" is a hyperweb, consisting of links between various segments of the data, annotations for preliminary interpretations, and link-types to indicate the nature of the relationship. Analogous to the convention of "view" in DBMS, this component helps the writer to sort out specific concepts in a collection of elemental inputs. At the resultant concept browser map, the user can click on any button and be taken directly to the linked statement in situ at the source materials or can navigate through the set of nodes and links.

Concept Synthesis: Continuing with the non-linear representation developed in the Cluster Browser, this tool fosters more comprehensive and interpretive manipulations. This exercise aims to consolidate ideas around central concepts extracted from the source data. The user is guided through exercises in observation, elaboration, and consolidation. The result is a set of notecards -- actually cells in a relational database -- that codify, extend, and/or arbitrate among the various source inputs. This workspace also serves as a brainstorming session in that the writer is encouraged to try out various permutations and elaborations on the core concepts.

While these aids foster exploration, they also focus the author's thinking. It is especially important to note that even at this relatively early stage in the "making of meaning," the writer can perform two powerful operations. She can return to the cluster browser at any given time and review existing concept maps or construct new ones -- thus viewing the problem space from a high level of abstraction. Or, she can move down to a more specific level to manipulate a set of notecards. Not only can the writer sort and filter the cards, she can try out different orderings and save each "trial run" as a separate file.

Information Threader: After a reasonable period of working with the Cluster Browser and its complement, the Concept Synthesizer, the writer may start to feel overwhelmed by the sheer amount of "views" (concept maps) and elaborations (files

of notecards) generated. The Information Threader begins the sculpting process for the final knowledge structure by coaching the writer to see potential patterns of meaning and to draw inferences from them.

The writer selects a categorical concept that seems to be of major importance in the belief structure constructed in the two previous phases. Prompts aid the writer to perform mental operations such as "comparison" and "contrast" or "inclusion" and "exclusion" in order to formulate an "issues" statement reflecting the concerns of these information clusters. Modeling the notion of basic inquiry, this exercise leads the writer to conflate -- using the dimensions of similarity and difference (comparison and contrast) as pruning criteria. The result is a collection of possible hypotheses, explanatory statements, implications, and/or inferences (which will -- in turn -- become thesis and/or topic statements for the text presentation).

Hierarchical planning: The Hierarchical Planner marks a major transition in the process modeled in the BROCA. It is a nexus at which the information structures woven in the "thinking and threading" segment must be reconceptualized to meet the requisites of linear text. Smith, Weiss, & Ferguson [Smith, Weiss, & Ferguson, 1987] discuss this change as a transition from a semantic net (essentially 3-D abstract structure) to a hierarchical outline (essentially 2-D concrete representation). This segment helps the writer to make this all-important transition from an implicit mental model to an explicit cultural artifact for a community of scientific scholars.

Though algorithms exist for transferring semantic nets into tree structures, a formalism which simply collapses the content and divests the context from the many complex judgments and intricate decisions may be too "ham-handed" for this application. BROCA uses a pruning algorithm to interpret the predicate expressions of the web. For example, if, in weaving the hypertext web of interconnections, the user has posted a "position" (a working interpretation) and has attached two arguments against the interpretation, linked by "refutes," the entity _refutes (A1, A2) might indicate a pro / con rhetoric strategy is emerging in this linked cluster. The resultant "outline" is therefore richer in meaning than simply conflating the web into a topical outline.

0rganization Mapping: After exploring the subject domain (the source data) and working out a richly interconnected belief structure drawn from these explorations, the writer must shift her attention to constructing a textual artifact that meets a set of external constraints and social expectations. Kopperschmidt [Kopperschmidt, 1985] characterizes this transition as a switch from cognitive macro structures to rhetoric micro structures. Similar to the drafting stage of writing, Organization Mapping helps the writer to focus more intensely on the requisites of the logical form and the conventions of scientific discourse.

Rhetoric and discourse studies have produced fairly detailed descriptions of the logical forms used in blocks of text (e.g., causal analysis, classification, comparison, definition, description, narration, and the like). The writer works not only with

organizational features and with expression and stylistics but also with such situation-specific concerns as audience analysis and purpose.

Revision Heuristics: The difference between copy-editing and revision is easily characterized. Revision usually refers to more substantial changes, such as improving style, adding to or subtracting from the content, rearranging parts, or completely writing. These more global, deep-structured revision activities are associated with higher-order cognitive skills (discerning patterns in bodies of information, exercising judgment, analysis, synthesis, and other metacognitive activities). Heuristic Revision comprises a suite of thinking frames for improving both coherence and expression. These heuristics are strategic (encouraging a re-thinking of high-level issues, such as purpose, point-of-view, audience analysis, voice, focus, and form) and tactical (including techniques of elaboration, such as level of detail, examples, support, flow, and balance).

BROCA uses a hybrid paradigm for interactive guidance. Part of the advice comes from adaptive tutoring using traditional AI formalisms and part of the mediating comes from the powers of reification (or representing complex processes as manipulable objects on the computer screen). Each of the six "tools" (1) accommodates deficiencies and thereby reduces frustration for the novice, (2) emulates some of the crucial functionality of traditional data structures and information forms, (3) enhances the environment and thereby sustains motivation, and (4) models robust expert behaviors.

2.4 Instructional Design and cognitive Architecture

While each of the six "tools" concentrates on a specific cluster of mental activities, all six have a unified method for delivering this layered instruction and a canonical architecture for the software and for the interfaces. Figure 3 gives an overview of the instructional framework and sequence of actions that structures each of the six tools.

Figure 3: System Overview of Hybrid Tutoring Capabilities

Area 1 -- Goal Setting and planning: Good thinking is mediated by having both a goal (desired outcome) and a plan (means of accomplishment). In multi-dimensional thinking, having an explicit, stable set of expectations fosters a kind of filtering activity that focuses the task from the outset. Each of the cognition facilitators handles the concretizing of goals in a slightly different manner; presentation and manipulation is appropriate to the phase of the process. Not only does this exercise help the novice focus on problem representation and sequence definition, this preliminary work "sets" the parameters of the adaptive tutor. Each tool now has a "frame" or backplane of conditions against which further actions can be evaluated during the remainder of the session on the tool. At this point, each tool tracks approximately 50 conditions. Clearly, the repertoire is rich, and becomes even richer as these preliminary combinations are supplemented with additional datapoints drawn from the user's subsequent activities in the microworld.

Area 2 -- Microworlds and Visual Algorithms: The second way in which BROCA teaches is similar to the cognitive "ecologies" advocated by Seymour Papert [Papert, 1980] and his LOGO worlds, where the child learner can practice profound concepts in familiar, manageable forms. (For example, Papert,s "turtle logic" enables the child to learn the conventions of programming by moving a cursor-turtle as an analog for

such relatively sophisticated manipulations as inclusion and exclusion.) Where possible, the interfaces of BROCA represent visual organizers for specific intellectual processes.

Like an adult version of "turtle logic," BROCA tokenizes mental manipulations and places the resultant visualizations in a constrained context so as to model the elaborations, state transitions, and reconfigurations of knowledge structures taking place in the problem spaces. Users interact with the computerized environment in rich but highly defined ways. The microworld mediates thinking by making choices explicit, by helping to manage the cognitive load, and by encouraging reflection during the thinking/writing process. In short, BROCA instantiates a "visual nomenclature" for reasoning whose components becomes synoptic overviews that trigger sophisticated intellectual activities such as formulating inferences about relationships, evoking strategies to facilitate thinking, and prompting metacognition, or self-regulation for deploying, adapting, or abandoning sets and subsets of strategies based on awareness of the situation.

Area 3 -- Diagnosis and Repair: Reasoning is a complex activity analogous to a contingency management problem. Even in scientific reasoning, for some "fuzzy" problems, only in working through candidate solutions does the nature of the problem become fixed, or even definable [Kuhn, Amsel, & O'Loughlin, 1988]. Rather than working in a linear fashion, good thinkers use an opportunistic approach. They constantly measure the emerging knowledge structure against a set of expectations, while at the same time recognizing and capitalizing on serendipitous gains, weaving these "discovered" possibilities into a new rendition of the overall plan and product [Hayes-Roth & Hayes-Roth, 1979].

Unfortunately, reasoning for novices is usually frail and one-dimensional; such impoverished capabilities do not lend themselves to interruptions or re-assessments. As evidenced by Bereiter and Scardamalia's research into the writing process for novices, little evidence can be found that weak writers can participate in self-cueing or self-monitoring activities while engaged in a production of text [Bereiter & Scardamalia, 1987]. In fact, the very act of breaking out of their one-dimensional, stream-of- consciousness mode jeopardizes the continued production of text.

Diagnosis-and-Repair is an evaluation loop that partners with the thinker to reduce the cognitive load and that encourages the student to enter into an strategy-driven assessment episode. This loop takes a very sophisticated, open-ended problem and pares it down to a manageable set of options for the inexperienced user. Succinctly, this facilitator operates in the following sequence. First, the user detects a mismatch between the goals (her intentions) and the on-going process of creating knowledge. As a response to this dissonance, the user requests help. Second, the system brings up a list of potential strategies applicable to the specific subtask the thinker is working on. Third, selecting any one of the suggested strategies brings up a focused workspace -- usually, a thinking frame that mediates the subtask. By presenting a limited set of options and by making suggestions (rather than dictating) about ways

to improve, the system both engages and challenges the thinker at the appropriate level.

Area 4 -- Adaptive Advice: In the metacognitive stage (diagnosis and repair), the machine partners with the user to develop the sensitivity and awareness necessary to know what strategy would work best in a given set of circumstances. Yet, because the diagnostic is performed by the user, there is a potential for a misjudgment. Additionally, if the system is to serve as an intelligent "guide" or "coach," the tools should have a feedback loop to indicate the "reasonableness" of the course of action the thinker is pursuing, basedlined against some known set of criteria.

Adaptive advice adjusts its statements based on an "intelligent" assessment of the situation -- meaning that the software compares the manipulation the user is working on with the conditions of the frame and determines how "correct" these actions are given the circumstances. The resultant prompting helps the user to learn the more subtle aspects of adapting to the requirements of the task. They also help the student to stay on the right track and avoid the frustration of pursuing a strategy whose results are later deemed to be inadequate to the task.

For all the intelligent tools of BROCA, adaptive help is generated through a kind of triangulation, based on the task situation (the conditions set in the frame) and the moves made by the problem-solver in the microworld or visual workspace. Monitoring the combination of specific task and place in the problem- solving process creates a lookup table for accessing instructional statements.

Area 5 -- Just-in-Time-Tutoring: While production skills and metacognitive skills are not interchangeable, they are correlated in that they must occur simultaneously in expert behaviors. After diagnosing a problem and getting a repair strategy, the student may still be at a loss as to whether the result measures up to professional expectations. Recognizing that users may need reminders of what a professional "end-result" looks like, we have embedded a set of models (extracts from publications deemed examples of excellence). Relevant portions of these models appear (appropriately correlated with the task the user is working on) with commentary pointing out the specific strengths of the presentation. This instruction (similar to a high-end form of context-sensitive help) is analogous to a job aid in that it gives a synoptic overview of the end-result the user is aiming for. Its purpose is to serve as a reminder or a refocusing prompt for the user rather than a full-blown instructional component. Though imitation, the user can incorporate the methods of the model into her own particular context and content.

2.5 System Overview

Commercial packages offering the writer a collection of tools (such as the analysis routines in the Writer's Workbench) have been around for some time now. Nevertheless, it is important to recognize that these tools are separate entities. While

the writer is free to pick and choose among them, the tools are not integrated nor are they supported by AI interpreters. In other words, work done with one tool does not translate seamlessly to the "world" of another tool. At a minimum, this is inconvenient. More telling for a worker or a learner, gains in one stage of composing are not easily consolidated and carried forward to the next stage. In fact, the welter of detail generated by some tools or heuristic routines may constitute a step backwards because the writer has to deal with (1) the cognitive overload of multiple versions or even contradictory instances of the same thoughts and (2) a potentially recurrent dis-integration of thoughts constructed while working with different tools or heuristic devices.

Figure 4 illustrates the "layered" nature of BROCA and shows how gains made in one workspace are passed on to the next. The complete system offers several "knowledge-weaving" paradigms, each designed to meet the requisites of a particular category of cognitive task and to exploit the talents represented by the particular user. Because the system is designed on a database paradigm, with each new "view" representing a value-added re-configuration of the data, it is possible to trace (via the links) higher-level propositions and inferences back to their inception or foundation in the source data.

Figure 4: Composite of BROCA's End-to-End Knowledge Development

3 Teaching Scientific Thinking Skills

Scientific reasoning requires that the thinker observe phenomena, perform elemental mental processing (e.g., detecting and classifying recurrent patterns), and then draw conclusions or explanations through higher-order cognition (e.g. inferencing and interpretation and application of enumerative generalizations). Evidence accumulating from the empirical study of scientific reasoning suggests that this ability to interpret does not map one-for-one on to discovery learning or native curiosity. While these facets are surely part of the motivational dimensions of human reasoning, the romantic version of the scientist making serendipitous "discoveries" is hardly the norm for this type of intellectual activity. Rather, scientific reasoning -- as practiced by an experienced professional -- involves a significant degree of awareness and control of the interactions between old knowledge (awareness of theory and expectations) with new information (situational observations). In short, the expert has both experientially derived mental models and metacognitive self-regulation to invoke, abandon, or creatively combine these problem-solving strategies. This expert level of insight has been built up over time and is a direct result of guided practice and painstaking mentoring.

BROCA performs the role of a mentor by mediating the process of scientific reasoning from beginning (data collection) to end (written report). Using visualizations of what might otherwise be opaque mental operations, the software guides the user through (1) collection/evaluation of evidence, (2) comparing data with theory, (3) formulating and testing hypotheses, (4) interpretation of results, (5) validation through replication. These complex mental activities take place within the framework of a second multi- dimensional, staged cognitive task -- that is, producing an acceptable piece of scientific prose (journal article, lab report, scientific journalism). Embedded within this authentic task are the profound acts of understanding necessary to transform observations into prose, or to move from data to knowledge.

4 Summary and Conclusions

This paper describes the conceptual design of BROCA, a computerized environment to merge the "language" of science with the manipulations, observations, and calculations foundational to scientific reasoning. Inherent in the discussion are three overarching assumptions about the design and development of advanced educational technologies. First, the design of automated systems should be rooted in cognitive science and educational theory. Advanced instructional technology holds great

promise; however, to truly move beyond pale imitations of existing educational media, designers must harvest the rich insights on how humans think and learn already established by cognitive psychology. Second, complex, multi-staged problem solving (in this case, scientific thinking) is an amalgam of visual and verbal symbol usage. By concurrently performing the manipulative/observational tasks along with the consolidative/interpretive tasks, the learner integrates both lower and higher level intellectual processes, as well as concrete and abstract forms of mentation. Third, highly sophisticated, interactive learning environments are possible without using the typical intelligent tutoring system (ITS) approach of constructing a student model, a domain knowledgebase, and an interpretive expert system. By employing cognitive task analysis, the system designer can model the process of the domain and embed adaptive help within the user's performance of the task. Such "cognitive tools" are just emerging as serious competitors to the more traditional approaches to adaptive pedagogy in interactive systems. We look forward to completing the development, running field evaluations, and reporting on the effectiveness of BROCA.

References

[Bereiter & Scardamalia, 1987]. Bereiter, C. & Scardamalia, M. (1987). The psychology of written composition. Hillsdale, NJ: Lawrence Erlbaum Associates.

[Bruner, 1971]. Bruner, J. S. (1971). The relevance of education. New York: W. W. Norton.

[Collins & Gentner, 1980]. Collins, A.M., & Gentner, D. (1980). A framework for a cognitive theory of writing. In L. W. Gregg & E. Steinberg (Eds.), Cognitive processes in writing (pp. 51-72). Hillsdale, NJ: Lawrence Erlbaum Associates.

[Garner, 1987]. Garner, R. (1987). Metacognition and reading comprehension. Norwood., NJ: Ablex.

[Gentner & Gentner, 1983]. Gentner, D., & Gentner, D. (1983). Flowing waters or teeming crowds: Mental models of electricity. In D. Gentner & A. L. Stevens (Eds.), Mental models (pp. 99-129). Hillsdale, NJ: Lawrence Erlbaum Associates.

[Gentner & Stevens, 1983]. Gentner, D., & Stevens, A. L. (1983). Mental models. Hillsdale, NJ: Lawrence Erlbaum Associates.

[Hayes, 1981]. Hayes, J. R. (1981). The complete problem solver Philadelphia, PA: The Franklin Institute.

[Hayes-Roth & Hayes-Roth, 1979]. Hayes-Roth, B., & Hayes-Roth, F. (1979). A cognitive model of planning. Cognitive Science, 3, 275-310.

[Holland, Holyoak, Nesbett, & Thagard, 1986]. Holland, J., Holyoak, K., Nesbett, R., & Thagard, P. (1986). Induction: Processes of inference, learning, and discovery. Cambridge, MA: MIT Press.

[Inhelder & Piaget, 1958]. Inhelder, B., & Piaget, J. (1958). The growth of logical thinking from childhood to adolescence. New York: Basic Books.

[Jones, Amiran, & Katims, 1985]. Jones, B.F., Amiran, M. R., & Katims, M. (1985). Teaching cognitive strategies and text structures within language arts programs. In J. Segal, S. F. Chipman, and R. Glaser (Eds.), Thinking and learning skills. Vol 1: Relating instruction to resarch (pp. 259- 295). Hillsdale, NJ: Lawrence Erlbaum Associates.

[Kieras, 1988]. Kieras, D. E. (1988). What mental models should be taught: Choosing instructional content for complex engineered systems. In J. Psotka, L. D. Massey, & S. A. Mutter (Eds.), Intelligent systems: Lessons learned (pp. 85-118). Hillsdale, NJ: Lawrence Erlbaum Associates.

[Kopperschmidt, 1985]. Kopperschmidt, J. (1985). An analysis of argumentation. In T. A. van Dijk (Eds.), Handbook of discourse analysis, Vol 2: Dimensions of discourse (pp. 159-168). London, UK: Academic Press.

[Kuhn, Amsel, & O'Loughlin, 1988]. Kuhn, D., Amsel, E., & O'Loughlin, M. (1988). The development of scientific thinking skills. New York: Academic Press.

[Lawson, 1983]. Lawson, A. (1983). Investigating and applying developmental psychology in the science classroom. In S. Paris, G. Olson, & H. Stevenson (Eds.), Learning and motivation in the classroom (pp. 113-135). Hillsdale, NJ: Erlbaum Associates.

[Luria & Yudovich, 1971]. Luria, A. R., & Yudovich, F. Ia.(1971). Speech and the development of mental processes in the child. J. Simon (Ed.), Baltimore: Penguin.

[McCormick, Miller, & Pressley, 1989]. McCormick, C. G., Miller, G., & Pressley, M. (1989). Cognitive strategy research: From basic research to educational applications. New York: Springer-Verlag.

[Miller & Stigler, 1991]. Miller, K. F. & Stigler, J. W. (1991). Meanings of skill: Effects of abacus expertise on number representation. Cognition and Instruction, 8(1), 29-67.

[Newell & Simon, 1972]. Newell, A., & Simon, H. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice Hall.

[Norman, Gentner, & Stevens, 1976]. Norman, D. A., Gentner, S., & Stevens, A. L. (1976). Comments on learning schemata and memory representation. In D. Klahr (Ed.), Cognition and instruction. Hillsdale, NJ: Lawrence Erlbaum Associates.

[Papert, 1980]. Papert, S. (1980). Mind-Storms: children, computers, and powerful ideas. New York: Basic Books.

[Pea, 1985]. Pea, R. D. (1985). Beyond amplification: Using the computer to reorganize mental functioning. Educational Psychologist, 20(4), 167-182.

[Perkins, 1986]. Perkins, D. N. (1986). Thinking frames. Educational Leadership, 43, 4 - 10.

[Perkins, 1985]. Perkins, D. N. (1985). The fingertip effect: How information-processing technology shapes thinking. Educational Researcher, (August/September), 11-17.

[Salomon, 1993]. Salomon, G. (1993). On the nature of pedagogic computer tools: The case of the writing partner. In S. P. Lajoie & S.J. Derry (Eds.), Computers as cognitive tools (pp. 179-196). Hillsdale, NJ: Lawrence Erlbaum Associates.

[Salomon, Perkins, & Globerson, 1991]. Salomon, G., Perkins, D. N., & Globerson, T. (1991). Partners in cognition: Extending human intelligence with intelligent technologies. Educational Researcher, 20(3), 2-9.

[Salomon, Globerson, & Guterman, 1989]. Salomon, G., Globerson, T., & Guterman, E. (1989). The computer as a zone of proximal development: Internalizing reading-related metacognition from a reading partner. Journal of Educational Psychology, 81(4), 620-627.

[Salomon, 1988]. Salomon, G. (1988). AI in reverse: Computer tools that turn cognitive. Journal of Educational Computing Research, 4(2), 123-139.

[Smith & Lansman, 1989]. Smith, J. B., & Lansman, M.C. (1989). A Cognitive basis for a computer writing environment. In B. K. Britton & S. M. Glynn. Computer writing environments: Theory, research, and design (pp. 17-56). Hillsdale, NJ: Lawrence Erlbaum Associates.

[Smith, Weiss, & Ferguson, 1987]. Smith J. B., Weiss, S. F., & Ferguson, G. J. (1987). A hypertext writing environment and its cognitive basis. Proceedings, Hypertext `87 Conference (pp. 195-214). Chapel Hill, NC: Association for Computing Machinery.

[Vygotsky, 1962]. Vygotsky, L. (1962). Thought and language. Eugenia Hanfmann & Gertrude Vakar (Trans.) Cambridge, MA: MIT Press.

[Zellermayer, Salomon, Globerson, & Givon, 1991]. Zellermayer, M., Salomon, G., Globerson, T., & Givon, H. (1991). Enhancing writing-related metacognitions through a computerized writing partner. American Educational Research Journal, 28(2), 372-391.