Application and Assessment of Cognitive-Dissonance Theory in the Learning Process

Abstract: The quality of an intelligent tutoring system is measured in terms of the speed and efficiency of learning. Many elements can improve this quality. If we consider the example of classical learning strategies, it is clear that they are not sufficient because the learner needs to be more involved in the learning session. Thus, there is a need for new co-operative learning strategies. For these strategies to be effective we need to be able to measure the weaknesses of the learner, and more specifically the discord in his or her ideas (internal conflict), in order to know which strategy is most suitable, when to use it, and what concepts need to be emphasised. Using the theory of cognitive-dissonance (discord between ideas), we have determined an indicator that measures the discord between the understanding of two elements of knowledge. To do this we have used the learning-by-disturbing strategy to test the confidence of the learner with regards to these units of knowledge and to make the learner aware of potential internal conflicts. We have developed a method allowing the detection of discordant concepts and the measure of dissonance rate. We also have shown that the learning process is improved when the tutor knows, for each learner, which concepts to focus.

Key Words: Intelligent tutoring systems, learning strategies, conflicts, cognitive-dissonance

1 Introduction

The goal of an intelligent tutoring system (ITS) is to reproduce the behaviour of an intelligent (competent) human tutor who can adapt his¹ teaching to the learning rhythm of the learner. Initially the control of training was assumed by the tutor (prescriptive approach), not the learner. More recent ITS developments consider a co-operative approach between the learner and the system which can simulate various partners such as a co-learner, a learning companion, a troublemaker etc., called pedagogical actors [Frasson and al, 96]. In fact, this evolution progressively highlighted two fundamental characteristics : (1) learning with an ITS is a constructive process [Frasson, Mengelle and Aïmeur, 97] involving several partners, called pedagogical actors, and (2) to improve learning, various strategies can be used such as one-on-one tutoring, learning with a co-learner, learning-by- teaching, learning-by-disturbing, and so on.

To make these strategies effective, we have to measure the weaknesses of the learner and more specifically the discord between the learner's ideas (cognitive-dissonance [Festinger, 89]), in order to know which strategy is best suited, when to use it, and which concepts need to be emphasised.

Being able to detect cognitive-dissonance enables the system to emphasise the concepts that the student has not mastered, and this will improve learning. This dissonance can be detected in all learning strategies but is most evident in the learning-by-disturbing strategy (see section 2.1.3).

In this article we show that we can detect and measure internal conflicts in a learner. To do this we deliberately provoke a debate between the troublemaker and the learner in order to test the latter's confidence in his knowledge and to make him aware of possible internal conflicts. The debate consists of a difference in opinion between the learner and the troublemaker (specialised tutor), and this difference is introduced to reach an obvious pedagogical goal: making the learner evaluate his own opinion and cognitive schemas and correcting his internal conflicts if necessary [Aïmeur and Frasson, 96; Aïmeur, Dufort, Leibu and Frasson, 97; Aïmeur, Dufort and Frasson, 97].

In order to do this successfully, we must detect in advance in the curriculum (i.e., the material to be taught) [Nkambou, Lefebvre and Gauthier, 96] the knowledge units which are likely to be misunderstood, and then, carefully plan the interventions of the troublemaker.

Three networks of knowledge (network of objectives, network of capabilities, and network of resources) form the curriculum. Some concepts are critical for the understanding of the course and a higher importance is assigned to them. In order to do this, we use the theory of cognitive-dissonance. In particular, we try to provoke a dissonance in the learner with regards to the critical concepts, and we observe what changes in attitude he adopts to reduce this dissonance. Faced with dissonant information the learner can keep his opinion and not be affected, in which case there is no dissonance. In other cases this can cause a dissonance, that is it can shake the learner's self-confidence and perhaps leave him open to being persuaded by the troublemaker.

This article examines several points: in section 2 we discuss ITS and we present the learning- by-disturbing strategy; in section 3 we deal with cognitive-dissonance theory, and clarify the principles that we use to determine a means of calculating a measure of the dissonance given three factors identified by Festinger. Finally we show why detecting cognitive-dissonance may help reinforce learning.

2 Intelligent tutoring systems

At the beginning of the eighties, ITS aimed to reproduce the behaviour of an intelligent (competent) human tutor who can adapt his teaching to the learning rhythm

of the student. Individualised teaching can be provided to the learner taking into account his previous knowledge, reactions, and progression throughout interactions with the tutor [Frasson and Gauthier, 90]. The control of the training is assumed by the tutor (prescriptive approach), not the learner. These systems were generally difficult to control from a pedagogical point of view and not very efficient. They were also complex to build, taking into account the multiplicity of types of expertise to incorporate (particularly the handling of the student model with its large amount of data and relationships) [Self, 88]. An inconvenience of most such ITS systems lies in their prescriptive approach, based on centralised decision-making and coaching.

More recent evolution of ITS development considers a co-operative approach between the learner and the system [Gilmore and Self, 88]. The system participates with the student in the learning process and facilitates knowledge acquisition through interactions under the control of the learner. Here, the learning process consists in the transfer of knowledge from the system to the learner in a tutoring session (a process that we also call knowledge acquisition), which is different from machine learning interpretation for which the system learns from user input. Intelligent tutoring systems include several components such as domain expertise, pedagogical expertise, and a student model. We will present them in a multi-strategic context as in the SAFARI project [Frasson and Aïmeur, 97] (see Figure 1 for the model):

• The curriculum : constitutes one of the essential components of an ITS and contains the domain expertise (the subject matter to be taught). This material is structured so that the system can easily extract the selected course or lesson.

Page 218

• The student model : reflects three facets of the profile of the learner : Emotional: includes aspects such as anxiousness, motivation, sociability, self-confidence, self-appreciation, learning preferences, and the like.
  Cognitive: represents the knowledge acquired by the learner with regards to the subject matter. Both correct knowledge and misconceptions are included.
  Inference: reflects the reasoning mechanism of the learner (deductive, inductive, analogous, ...)
• The planner : decides dynamically which lesson or which course is most adapted to the learner taking into account the student model and the time available in the learning session.
• The pedagogical model : executes a learning strategy based on the co-operation of several pedagogical actors such as a tutor, a co-learner, a companion, a troublemaker, etc. [Frasson, Mengelle, Aïmeur and Gouardères, 96; Frasson, Mengelle and Aïmeur, 97]. These actors can play various roles depending on the learning strategy in which they are involved.
  For example, the tutor can act as a coach during a one-to-one strategy, as a referee in the learning companion strategy, or as a conspirator in the troublemaker strategy.
• The session manager : is responsible for the proper execution of a learning session. It takes important tutoring decisions and controls the learning. In order to do this, it asks the planner for a lesson, chooses the appropriate resources and activates a learning strategy that will allow the student to reach a given pedagogical goal.
• Didactic resources : correspond to tactical means necessary to ensure the teaching of a given subject matter. They include demonstrations, exercises, problems, multimedia documents, HTML, etc. They are activated by the session manager to whom they report.
• The interface : supports the interaction between the learner and the other components. It should be as ergonomic as possible in order to captivate the learner.
• The learner : Depending on the ITS, the learner can consult his model or choose a pedagogical strategy.

2.1 Co-operative Strategies

The principle of co-operative tutoring systems (also called social learning systems) is based on the use of the computer not as a directive training means but instead as a way to exchange, control and build knowledge. Several experiments have shown that two persons working together will learn more than in individual training. Constructivist approaches assess that the learner builds his own knowledge using

his/her experience and interaction with the real working environment. Learning in context states that knowledge construction results from a common interaction with the real world (including not only specific aspects of the domain but also social, cultural, and historical aspects) using the context [Clancey, 92].

In that sense, several models have been developed which generally are called social learning systems, co-operative systems, or collaborative systems. If both co-operative and collaborative systems can be considered as social learning systems, there is a difference between co-operation and collaboration. Collaboration requires a joint action of the participants and a mutual understanding of the task to execute, each participant having his or her own objectives. Co-operation requires sharing the responsibilities between the participants for executing a task and the knowledge of mutual objectives [Baker, 93]. In both cases social agents can be computer simulated or real humans sharing a single computer or distributed on a network of computers. Also, the role of the learner and the teacher can be interchanged, and this aspect provides a variety of learning strategies that we review in the following section.

2.1.1 Directive Learning

This approach (also called the one-on-one strategy) [Sleeman and Brown, 82] preceded the co-operative systems and consists of having the computer simulate an intelligent tutor who can understand the learner and provide adaptive tutoring. The learner receives knowledge directly from the tutor, who communicates and acts according to a prescriptive behaviour.

2.1.2 Peer Learning

Co-operative learning systems adopt a constructive approach using the computer more as a partner than as a tutor. Multiple agents that are either computer simulated or real human beings can work on the same computer or share a computer network.

Chan and Baskin proposed a three-agent learning situation [Chan and Baskin, 90] which consists of a co-operation between a human learner and a simulated learning companion. They learn together under the guidance of the tutor. The companion and the learner perform the same task and exchange ideas on the problem. The learner and the co-learner (the companion) work together and ask the tutor for help only if they cannot find a solution.

The learning-by-disturbing strategy [Aïmeur and Frasson, 96; Aïmeur, Dufort, Leibu and Frasson, 97] suggests that the computer can simulate two agents: a tutor and a

troublemaker. The level of competence of the troublemaker is superior to that of the learner in order to provide reasonable competition. In addition, it has some pedagogical knowledge, which can help it to plan its interactions efficiently.

For the strategy to be pedagogically sound, the troublemaker proposes erroneous suggestions to the student emphasising some of the finer points of the exercise at hand.

2.1.3 The Learning-by-Disturbing Strategy

In this section we describe the learning-by-disturbing strategy by describing the participants and their roles, comparing the strategy with that of the peer learning.

Description:

The learning-by-disturbing strategy implicates three participants :

• The tutor : presents to the team of students both the lessons and the exercises to be solved. It is the tutor which controls both the content and the length of the session. At any time, the tutor may intervene to help one of the students in the task, and finally, evaluates the performance of the learner.
• The learner : is the human student who is using the ITS. The learner interacts with the other participants via either pseudo-natural language or symbolic dialogue. The system maintains at all times a model of the learner which describes the state of the student's knowledge relative to the system's objectives and the student's emotional state. The latter is particularly relevant to the troublemaker strategy since it is important to gage the student's confidence levels to plan the troublemaker's actions.
• The troublemaker : appears to be a simulation of a student working with the learner. In fact the troublemaker possesses both pedagogical expertise and a level of knowledge of the domain comparable to that of the tutor. The troublemaker uses this pedagogical expertise to maximise the impact of its interventions. The role of the troublemaker is to unsettle the student by proposing solutions which are sometimes truthful but other times erroneous. This tests the student's self-confidence and obliges him to defend his point of view. We believe that, in certain conditions, this argumentation increases the student s motivation and increases learning.

The reader may ask why the tutor does not ensure both the teaching and the trouble-making functions. The answer is clear. In the framework of intelligent tutoring systems, one cannot afford to have the student lose confidence in the tutor. In fact, the troublemaker, by making suggestions that are sometimes correct but also sometimes erroneous, will inevitably lose credibility in the eyes of the student. We present the troublemaker as a student who will work with the learner without revealing its true intentions. The learner will never know that the troublemaker is in fact a tutor with a specialised role: that of testing and provoking the student.

2.1.4 Qualitative Comparison between the "Learning Companion" Strategy and "Learning-by-Disturbing"

The learning-by-disturbing strategy is relatively new and is still under development. Those who are accustomed to the learning companion strategy may well ask themselves: why the learning-by-disturbing strategy is necessary? One justification is given in [Aïmeur and Frasson, 96]: there is a need to test the self-confidence of the learner, to introduce a new form of motivation, to increase the degree of stimulation, and to reinforce the knowledge of the learner.

However, each method has advantages and weaknesses. To appreciate more precisely their differences we will consider some areas in which innovative work has been done which improve the efficiency of an ITS. They concern the self-confidence of the learner, his motivation in learning, and the pedagogical knowledge implied. In the following we briefly review the form of these criteria in the two strategies: the companion and learning-by-disturbing.

• Learner's self-confidence
  With the learning companion, the learner needs to be self-confident in order to discuss with the companion. Learning-by-disturbing forces the learner to be even more self-confident in his actions or conclusions and to distinguish between correct and incorrect solutions. In addition, it strengthens the knowledge acquisition process. The learner confronts the troublemaker, facing its position and needing to prove that he has learned correctly. Ultimately, he might feel some pleasure in showing his capacity in front of the troublemaker.
• Motivation in learning
  With the companion, although an evaluation has to be done by the tutor, the motivation is based on a feeling of emulation. As we have mentioned earlier we need to know the self-confidence of the learner, to introduce a new form of motivation, to increase the degree of stimulation, and to anchor the knowledge in the learner. We can also make a link between the learning-by-disturbing strategy and the argument-teaching method [Schank & Jona, 91]. In both methods, the presence of controversy and the discussions that follow (between co-learners) have a positive effect on learning. The idea that discussions in a group-learning situation increase motivation is not new. Roschelle [Roschelle, 92] remarked that "Piaget and his followers tended to see collaboration as producing productive individual cognitive conflict - disequilibrium drives conceptual change."

• Pedagogical knowledge
  Unlike the learning companion, the troublemaker possesses pedagogical knowledge. Despite the fact that it appears to be a student, in this respect it is acting as a tutor. Two points are to be noted:

Page 222

Both the troublemaker and the tutor have complete knowledge of the domain. This is not necessarily the case for the learning companion. In addition, the troublemaker possesses certain pedagogical knowledge that the tutor does not have: When to disturb ? How far to argue an erroneous point ?

We will use cognitive-dissonance theory to show how the troublemaker strategy can be effectively used. Before this, we will introduce the different types of conflict that can occur in an ITS.

2.2 Conflict in ITS

Over the last few years, co-operative learning systems [Slavin, 90] have been extensively studied in different domains both in terms of their design and in terms of their implementation. Since in such systems there is an interaction and a dialogue between several partners, it is inevitable that there should arise conflicts between them.

Researchers have asked many questions about these conflicts including: What to do in case of a conflict? Can one predict and avoid conflict? How does one resolve the conflict? Can conflict be quantified? In our opinion, although research has been done in the area of conflict resolution, not enough work has been done in profiting from the conflicts which do arise.

There exists several types of conflict in ITS:

internal conflicts in the learner model (knowledge poorly acquired, missing knowledge, etc.),

external conflicts between the learner and different participants in a learning session,

external conflicts between tutors delivering the course or designers of a single course in a multidisciplinary setting,

external conflict between designers of the ITS from the architectural point of view, since the degree of constraint between the learner and the tutor is a deliberate pedagogical choice.

In this section we have highlighted the different types of conflict that can occur in ITS. In the discussion that follows, we are interested only in the first two types of conflict. We will now examine internal conflicts. In particular we will use the theory of cognitive-dissonance to explain how one can identify and quantify conflict.

3 Cognitive-Dissonance

Each of us memorise at a given moment a certain number of facts, both truth and false, partly true or partly false, concerning ourselves or others. Social psychologists of the 50s called these facts cognition. This cognition refers to conscious representations of fact in our mind. They can be concepts, ideas, knowledge, opinions, beliefs, etc. They can refer to one-self ("I am interested in computer science"), to one's behaviour ("I am waiting for Suzanne"), to one's social environment ("my

neighbours are fighting again") or to one's environment ("the sky is blue"). Most of the cognition that constitutes our cognitive environment is not independent of each other. On the contrary, they are related in ways that can be perfectly harmonious but might also be quite uncomfortable.

Between 1955 and 1960, several psychological theories appeared. Their main goal was to explain the relations between cognition, and in particular how these relations were built and adjusted. The most important of these theories were the theory of cognitive-dissonance [Festinger, 57], Heider's theory of equilibrium [Heider, 58] and Osgood and Tannenbaum's theory of congruence [Osgood and Tannenbaum, 55]. These theories are traditionally grouped in one paradigm, that of cognitive consistency, since they all describe an organisation of cognition. More specifically these different theories suggest that when cognition are not well linked, in other words, when they are not well organised, a cognitive readjustment has to be done to re-establish a more harmonious organisation.

Cognitive-dissonance is a theory originally developed by Festinger which had a great impact on the social psychology community. According to this approach cognition represents an element of knowledge. Cognition can be, with respect to each other, in either a relevant relation or an irrelevant one. When they are in a relevant relation, they can interact, imply each other, contradict each other, or contribute to each other. The theory is only interested in cognition which are in a relevant relation with each other and these can either be consonant (consistent) or dissonant (inconsistent). More formally, if x and y are cognition, then they are in a consonant state if x implies y or if x contributes to y. They are in a dissonant state if x contradicts y. Like the motivational states of hunger or thirst, the state of dissonance is unpleasant and prompts the individual to attempt to reduce that dissonance.

3.1 Example

Consider someone who buys an expensive car but discovers that it is not comfortable on long drives. Dissonance exists between his beliefs that he has bought a good car and that a good car should be comfortable. Dissonance could be eliminated by deciding that it does not matter since the car is mainly used for short trips (reducing the importance of the dissonant belief) or focusing on the car advantages such as safety, appearance, or handling (thereby adding more consonant beliefs). The dissonance could also be eliminated by getting rid of the car, but this behaviour is much harder to achieve than changing beliefs.

3.2 Definition

Festinger's definition of cognitive-dissonance is the perception, by a subject, of a difference, of variable intensity, between what has been previously perceived and learned and new information. This process is illustrated by Figure 2.

Figure 2. Steps in the cognitive-dissonance process [Dufort, Aïmeur and Frasson, 97].

Festinger adds that, essentially, inertia makes us accept what we believe to be true. Nevertheless, there exist situations when we are exposed to contradictory information. The feeling of cognitive-dissonance so triggered will start the process illustrated in Figure 2.

Festinger strongly links cognitive-dissonance and internal motivation: "The existence of dissonance, being psychologically uncomfortable, will motivate the person to try to reduce the dissonance and achieve consonance. In short, I am proposing that dissonance, that is, the existence of non-fitting relations among cognition, is a motivating factor in its own right." [Festinger, 57].

A key feature of Festinger's theory is the expectations that the subject has. In fact, the subject seeks to corroborate his conception of the environment by what he perceives. "New information may become known to a person, creating at least a momentary dissonance with existing knowledge, opinion or cognition concerning behaviour. Since a person does not have complete and perfect control over the information that reaches him and over events that can happen in his environment, such dissonance may easily arise." [Festinger, 89].

An individual experiencing cognitive-dissonance may lead to negative consequences:

• The individual may incorporate inconsistent and contradictory knowledge into his cognitive schemas and then use them in a dysfunctional manner. This is exemplified when a student, who believes from childhood that two objects of unequal mass will fall at different velocities, is taught the contrary. It is possible that this student will maintain both beliefs, being able to state the law of physics correctly but answering incorrectly on an exam. (It is important to note that a single piece of information can be represented in different manners and then

  Page 225

  stored as different cognitive schemas. This poor assimilation of information is precisely the type of error that an ITS can detect and correct: see step 2 of figure 2.)

• The individual may attempt to avoid the situation which has caused the dissonance, even if this means committing an error. This reflex is instinctive, unconscious, and depends on the personality of the subject and his perception of the resources available to him.
  For example, a student who does not understand a subject may decide to no longer study it unless forced to. Another student in the same circumstances may simply refuse to ask questions in class for fear of being mocked.
• The individual may become suspicious of new information and therefore his confidence when interacting with others may diminish.

Therefore, all individuals will experience cognitive-dissonance while interacting with their environment. A very common source is the interaction individuals have with other people: "When a person is confronted with an opinion contrary to his own which is held by people like himself, he experiences dissonance" [Festinger, 89].

The intensity of such dissonance depends on two factors:

• The perceived competence of the person or group expressing the contradictory opinion (in our case this is the perceived competence of the troublemaker), and
• The emotional relationship to the person or group expressing the contradictory opinion (in our case this is related to the emotional relationship with the troublemaker).

The individual experiencing cognitive-dissonance triggered by another person can react in four ways:

1. Dismissing the subject of dispute as being unimportant.
2. Dismissing the other person as being unimportant.
3. Attempting to eliminate the dissonance by changing his own opinion (by letting himself be convinced) or by attempting to change the opinion of the other person (in particular by initiating a debate with that person).
4. Seeking new information in his environment which would support his opinion. For example in a community (such as a system with several participants) the individual could seek social support.

Each of the theories previously cited (cognitive-dissonance, Heider's theory of equilibrium and Osgood and Tannenbaum's theory of congruence) has specific aspects that make it inapplicable to certain domains. We have paid particular attention to the theory of cognitive-dissonance because it is the one that allows us to best understand the internal conflicts that exist in the learner's mind and that best explains the importance of the troublemaker in the learning process. In fact, we believe that the troublemaker strategy is an ingenious way to detect internal conflicts and to make the

learner aware of them. By provoking external conflicts between the learner and the troublemaker the latter must react to rectify an uncomfortable situation.

3.3 Learning-by-Disturbing as a Way to Correct Cognitive-Dissonance

The following points describe the learning-by-disturbing strategy in the context of cognitive-dissonance theory :

1. A cognitive-dissonance is triggered by the troublemaker's interventions;
2. At that time, the troublemaker is the only available source of information²;
3. In order to reduce the dissonance the learner is motivated to search for new information in his environment;
4. The mechanisms used are dialogue and debate with the troublemaker, and this process has two outcomes : the student can let himself be convinced, or the student can change his environment by convincing the troublemaker.

Finally, two factors influence the outcome of this debate:

• The confidence that the student has in his cognitive schema, and
• The ability the troublemaker has to express its ideas in a convincing manner.

It is interesting to ask what impact a given intervention of the troublemaker should have. It is clear that cognitive-dissonance should not be the result each and every time, so when is it important to disturb the student? A few important points to keep in mind are:

• If the student's confidence is dropping, it is interesting to have the troublemaker present correct suggestions to reinforce the student's beliefs.
• In some cases the troublemaker can make such a serious error that there is no doubt that the student can correct it. This will increase the student's confidence and give him a feeling of competence.
• When the student begins to develop self-confidence, the troublemaker's suggestions should become more aggressive in order to disturb the student. At these moments the tutor can intervene to demand consensus so that the student does not dismiss the troublemaker out of hand.

² One can, however, imagine a strategy in which the tutor is accessible during the debate between the student and the troublemaker.

3.4 A Note on Measuring Cognitive-Dissonance

First we need to clearly specify what we mean by measuring cognitive-dissonance. This is a difficult thing: "Localising a gap in someone's knowledge is difficult." [VanLehn, Jon and Chi, 92]. However, it is necessary to clearly express what it is that we are calculating.

Joule [Vallerand and Thill, 93] showed that the behaviour of an individual has a specific role in the interpretation of Festinger's theory. It is this behaviour that is the origin of dissonance and that guides each stage of cognitive-dissonance. A number of specific observable behaviours are associated with any given stage whose key is cognitive-dissonance. For example, an individual feeling guilty when smoking may light a cigarette but put it out after a few puffs. An active means of observing the individual is to ask him questions. "Do you believe that smoking is bad for your health?" and "Do you want to stop smoking?" If the individual answers respectively yes and no, then we can affirm that we have detected cognitive-dissonance.

Proposition 1: The steps of the method an individual uses to solve a problem or the answers to a test constitute behaviours from which we can detect cognitive-dissonance.

In order to accept this proposition we have to privilege lax interpretation of Festinger's theory which considers that a subject is in a state of cognitive-dissonance whenever two of his cognition are in a dissonant state. This allows us to consider the set of cognition of an individual as potentially dissonant. The other interpretation of the theory, the radical one, proposes a more restrictive approach which draws a link between cognitive-dissonance and a state of tension; here the distinction is between cognitive-dissonance (existence of tension) and incoherence (state of ideas).

Proposition 2: Any related cognition that is in a state of incoherence is considered dissonant, independently of the state of tension present in the individual.

Of course it is not possible to detect all of the cognitive-dissonances present in an individual or even to confirm with great certainty that what was detected is in fact cognitive-dissonance. As we will see later, we can only detect part of existing dissonance with tests; this part varies with the raw score obtained on the test. Even so, we believe that the result can in most cases guide interventions of the pedagogue or the tutoring system.

Finally, it is important to specify that when Heider and his successors [Morissette, 58] allow themselves to quantify the total rate of disequilibrium in the individual, they add the disequilibrium emanating from ideas that can be unrelated. Knowing that an individual presents a disequilibrium in his aversion of classical music and a disequilibrium in his passion for ice cream will tell us nothing. Someone believing in Festinger's theories might say "One shouldn't add apples and oranges". In contrast, Festinger calculates the global rate of dissonance from a single cognition and those that are directly related to it. Our approach is situated between these two extremes.

Our use of a single knowledge structure on a restricted subject avoids comparing incompatible things (in doing so we agree with Heider's critics). Manipulating the knowledge structure as a whole allows us to compare indirectly related cognition and in this sense we differ a little from Festinger's approach.

Proposition 3: A knowledge structure can correspond to a schema of thought as defined in Festinger's theory. Each item of the structure can be dissonant when compared to another item as long as a link can be drawn from one to the other.

3.5 Developing a Methodology to Perform Cognitive-Dissonance Measurement

According to Festinger [Festinger, 57], if we wish to measure the amplitude of a dissonance we must take into account three factors:

1. If two cognitive elements are related, the relation between them is either dissonant or consonant.
2. The magnitude of the dissonance (or consonance) increases as the importance or value of the elements increases.
3. The total amount of dissonance that exists between two clusters of cognitive elements is a function of the weighted proportion of all linked relations between the two clusters that are dissonant.

Therefore if we wish to quantify the cognitive-dissonance in a learner, we must keep into account the importance of the elements of cognition which are in conflict. We must also keep account of the relationship between these two elements. If the two elements are weakly linked, the amplitude of the dissonance must also be weak. There cannot exist a dissonance between elements that have nothing in common. What Festinger s theory does not indicate is exactly how one calculates a value to measure dissonance.

In recent publications, several methods have been proposed. Vallerand [Vallerand, 94] proposed a simple formula where the dissonance value is given by the sum of the dissonant cognition divided by the sum of the cognition (both dissonant and consonant). The total cognitive-dissonance (CD_total) is therefore given by:

In the realm of computer science, formulas giving a result between 0 and 1 are easy to use since the resulting value falls in a standard range. For this reason, this formula is good. On the other hand, this formula does not take into account the importance that the individual associates to each cognition. Another, more complete, formula is proposed by Joule in [Vallerand and Thill, 93]:

This time, the formula does take into account the importance that is associated with each cognition but no longer returns a value between 0 and 1. Neither of these two formulas is entirely satisfactory from the point of view of ITS since neither the importance nor the nature of each cognition is known. How does one quantify what is happening in the mind of the student?

In experimental psychology, a researcher can evaluate different psychological parameters (including cognitive-dissonance) in an individual or in a target public (the group to whom the course is destined). This can be done through interviews and by analysing the results of questionnaires. Traditional methods, such as interviews, require both time and resources. As we will see later, identifying potential dissonant elements in a learner can help identify the important points of a curriculum, the points where a strategy should focus. In particular, in the learning-by-disturbing strategy these critical points are those in which the troublemaker's actions and arguments must be well developed. In fact, these are the points which the learner may not acquire properly. It is normal that the domain expert should emphasise these points.

Let us now elaborate further the notion of a target public. The population for which a course is constructed is usually fairly homogeneous. Among other common traits, we can expect the individuals to have similar background knowledge, comparable cognitive characteristics and similar objectives. In order to evaluate the students in the target public we first give them a preliminary questionnaire to test their knowledge of each element of the curriculum (the subject matter to be taught). This questionnaire will give indications as to the composition of the target public. For example, these results may indicate that the target public is heterogeneous and it may be preferable to create several more tailored courses, better adapted to the newly identified sub-groups.

The measure of cognitive-dissonance, in conjunction with other measures, will give indications as to:

• The amount of miscomprehension of the domain in an individual (which we compare to a measure of the entropy in ideas). Learning knowledge as isolated fragments can lead to a poor comprehension of that knowledge. Raw test results do not give a clear measure of this problem.
• The amount of miscomprehension of a specific capability in comparison to the other capabilities in individuals in the target public. This is a particularly interesting result since it allows us to identify capabilities in the curriculum that are more likely to be misunderstood.

Let us examine more closely the structure of the curriculum. Many different models of curriculum have been developed to provide support to various ITS using them [McCalla, 90]. The structures used vary in size and complexity. Although the method exposed hereafter can be adapted to be used with any structure (such as the concept network), we will use a model of the curriculum similar to that proposed by Nkambou [Nkambou, Lefebvre and Gauthier, 96].

The network represented in Figure 3 has been employed to model the use of the Baxter pump. This pump is used in medicine to administer perfusions. For example c₁ corresponds to the capability "infusion rate" and c₁₆ corresponds the capability "primary infusion". Since both prerequisite and contribution links can be found between c₁ and c₁₆ we can say that c₁ is a prerequisite to c₁₆.

Transition nodes correspond to didactic resources. For example, T₁ groups the demonstrations and the exercises that discuss "programming (infusion rate)". Since our calculation only considers capabilities, we do not discuss the transition nodes.

Since the structured representation of ideas in the curriculum corresponds to an ideal thought schema, we assume that it does not contain any contradictions and thus no cognitive-dissonance. Analysing the results of the learner through a pre-test, using the curriculum as a reference, allows us to make inferences about the organisation of ideas in the learner's mind.

In order to reach our goal of calculating a value for cognitive-dissonance, we must associate a numerical value to each node in the graph indicating its importance. There is no method which will allow us to obtain the importance the learner attributes to each capability since it is a subjective thing. The importance that the tutor associates with a capability while he gives a course can influence the importance that the student attributes to that capability. We can give the nodes values that represent the

importance of the capability in the overall course. We suppose that this value reflects to some degree the value that a learner attributes to the capabilities. In order to do this we define the following attribute:

The values given will be between 0 (not necessary) and 1 (crucial). An indication of the importance of a capability in the curriculum is the number of references made to that capability. Let us define now a function d that gives the distance between two capabilities. This function must take into account the structure of the curriculum and the types of links that are used.

We can give some constraints on the calculation of d:

• Each link type must have a numerical value of 1 or greater associated to it (see figure 3 for types and figure 4 for values).
• If c₁ leads to c₂ the distance d is calculated as follows: for each possible path, find the greatest value between c₁ and c₂, and take the minimum.

The next step is to calculate the relatedness (R) between two cognition. The basic cases are obvious: for two cognition, which are completely unrelated, the function R must give 0. The measure of relatedness is at its maximum when comparing a cognition to itself, in which case R returns 1:

• This definition of R satisfies the condition that relatedness is maximal when the distance is 0 (i.e. when one compares a cognition to itself).
• Since the curriculum is an oriented graph, in general R(c₁,c₂) R(c₂,c₁).
• The form of the formula (1/x) ensures that the function is bounded; in fact, since there are only four types of links, the limits of the function are known.

Figure 4 gives an example of a network.

In this example, to calculate d(c₁,c₇) we find three possible paths whose most costly link is 4, 4, and 3 respectively. We therefore choose 3 as a value for the distance between c₁ and c₇ and the relatedness R(c₁,c₇)=0.25.

In order to justify this method we note the following:

• The number of links between two capabilities depends on the granularity of the curriculum and so summing the values along a path is not a viable solution.
• The weakest link in a chain represents its overall strength. For example, if in a chain we find even one link of preferred prerequisite then we know that the relationship between the capabilities is weak.
• In all the possible paths, choosing the most advantageous consists of choosing the one with the highest relatedness.

According to cognitive-dissonance theory, cognitive-dissonance can occur whenever two cognition, beliefs or behaviours related to the same cognitive schema are in conflict. In the case of the curriculum, we can detect a dissonance if the learner has correctly answered a question on capability c₁ but has also incorrectly answered another question on c₁ or on c₂ which is prerequisite to c₁. In other cases it is not possible to detect dissonance. We cannot for example detect apparent consonance³. An apparent consonance can occur when the learner errs on two questions, both related to capability c even though both errors were due to dissonant beliefs.

In order to give a concrete example let us suppose that the learner answers questions on elementary physics. The questions are true or false. The student answers: "There is air on the surface of the Moon. (False)" and "You can hear an explosion on the surface of the Moon. (False)". There seems to be consonance between these results since the student has succeeded in answering two questions related to the same capability. Despite this, if the learner had answered the first question correctly because he knows that there is no air on the Moon but answered the second false because he believes that there can only be explosions on Earth, then perhaps there is dissonance. Perhaps the student believes that sound can travel without the support of air and this dissonance has not been detected.

The calculation of the potential dissonance between two capabilities is a function that takes into account both the importance of the capabilities and their relatedness. The function CD_pot is calculated as follows (with a max function since we wish to obtain the maximum dissonance possible):

In order to evaluate all the students who are in the target public, the expert must present to them a preliminary questionnaire which tests their knowledge on each element of the curriculum. This questionnaire will give indications as to the composition of the target public. For example these results may indicate that the target public is very heterogeneous and that it is preferable to create several more tailored courses, better adapted to the newly identified sub-groups.

³ One can also see this as a hidden dissonance.

The questions can take many forms (true or false, multiple-choice, associative, ...). In order to simplify the calculation we will suppose that each question is a "true or false" question. The results are easier to analyse than for multiple-choice or associative questions. Each question is taken from one or more elements from the curriculum. Let Q be the set of questions. The relation S_c gives for a question q its corresponding capability in the curriculum:

Let V(q) be the result of the question from a given learner (0 = failed; 1 = succeeded). The success difference between two questions q₁ and q₂ for a given learner i is given by:

We consider that there is cognitive-dissonance when a question has been successfully answered while a prerequisite to that question (or to a question related to the same capability) has not. The success-difference function is therefore not symmetrical. This function becomes more complicated when one considers multiple-choice questions. Multiple-choice questions are often related to multiple capabilities and thus when the learner fails such a question the diagnosis can be very complex. Nevertheless, analysing these results would be a great achievement; developing a methodology to do so, is in our opinion, an important field of research.

We can calculate the average of the success difference for a pair of questions for a group of students of a target public. We must ensure that the target public is sufficiently uniform, otherwise the value is not usable. This value D_mean, can be used instead of D_i in the following formulas. If the target public contains N students then:

If we wish to calculate the total dissonance in a learner i, within the framework of the capabilities of a curriculum, we can use the following formula:

If we wish to calculate the average cognitive-dissonance for a given capability in a single learner or in the learners of the target public we can use the following formula:

The quantity CD_capability gives, for each capability in the curriculum, a measure of cognitive-dissonance associated with that capability. This calculation takes into account the links existing between the particular capability and all other capabilities. In addition, it considers pairwise all the questions which are related to that capability. A high value indicates that the capability is likely to be misunderstood by that individual. Because of this, it becomes important to develop the interventions of the pedagogical actors (tutor, troublemaker, etc) in these critical points of the curriculum. Identification of these critical points accelerates the development of the curriculum by allowing the expert to concentrate the development effort in key areas.

4 Application to an ITS in the Medical Domain

The task we are modelling is medical diagnosis, more specifically the diverse illnesses affecting breasts. The central element is a set of four mammographs. We have chosen relatively simple cases where the student needs to consult the medical history of the patient, but where the breast radiographs can present no more than one pathology.

The exercise is divided into four parts:

• Ordering the mammographs: by using a series of image manipulation tools (found on the toolbar), the student must place the four mammographs in the correct order and must orient them properly. The orientation of the radiographs supposes that the student can manipulate the images by rotating them or flipping them both horizontally and vertically.
• Identification of critical regions: the learner traces the contour of regions of interest for the diagnostic of the pathology.
• Region characterisation : at all times, the student may select a region and associate to it a feature.
• Choice of diagnostic: the student chooses the diagnostic that he wishes to propose from the left-hand list in an order that indicates their relative importance and sends these items to the right-hand list. He can also remove an item from the right-hand list.

The system is composed of a window containing the four mammographs, the list of possible features and a list of possible diagnostics. The two actors, the tutor and the troublemaker, each have their own dedicated window so that they be well dissociated from the exercise. These two actors cooperate in their teaching task by planning their interventions; each intervention is negotiated with the other actor in order to create for the learner a pedagogical environment that is dynamic and stimulating.

The tutor presents each exercise and makes comments so as to guide the student in the resolution of the exercise. He can correct the learner in either a weak manner, by telling him that an error was committed and letting him correct it by himself, or in a strong manner, by correcting the error and presenting the solution to the student. The troublemaker is free to intervene at any moment to give advice (either truthful or false). The tutor can ask for a consensus between the learner and the troublemaker. Figure 5 shows a glimpse of the system, at the beginning of the exercise.

4.1 Curriculum for the Radiology Course

The curriculum representing the capabilities necessary for the resolution of this exercise is relatively simple. It is always possible to increase the granularity of the

curriculum so as to develop exercises which focus on more specific capabilities. Figure 6 presents this curriculum.

In order to show an application of formulas used to calculate dissonance, we have obtained results for a questionnaire of 20 questions on the curriculum shown above. In the present case, the score of the learner is 60 %. It is important to note that as the score approaches 50 % it is easier to detect dissonance, as figures 7, 8, and 9 show. If the score is too high or too low, apparent consonance appears more often. Table 1 shows the questions, the corresponding capabilities, and the results.

Based on the curriculum graph we fill out a table of relatedness (table 2) for each pair of capabilities. For example for the capabilities c₁ and c₈ we obtain two possible paths. The first is c₁ T₁c₄T₃c₈ and its worst link has a value of 1. The second is c₁ T₁c₄T₄c₈ and its worst link has a value of 3. We therefore choose the first path. In this case

The calculation of cognitive-dissonance gives a value of CD_total(i)=0.252743, which is an average value. If we study in detail the results of the pre-test, we can see that for six out of ten capabilities there is a strong difference between the results of corresponding questions; therefore a strong cognitive-dissonance. We can calculate the cognitive-dissonance for each capability for this learner:

What table 3 shows is that this individual is likely to misunderstand the capabilities c₂,c₃, c₆ and c₇. The expert, conscious of these results, must develop more carefully the interventions of the actors for these capabilities. The material from the course must also emphasise these capabilities in addition to the questions erroneously answered by the learner. Combining these results with the pre-test results and other statistical analysis, the expert can better predict the needs and the behaviour of the student when faced with this learning material.

4.2 Analysing the CD_total Formula with a Probabilistic Approach

In order to study the behaviour of the formula allowing the calculation of CD_total, we have generated 3500 results such as those found in Table 1. We have used a uniform rule to generate the total score (out of 20). The goal of this exercise was to verify several hypotheses which, while they intuitively seem to be true, must be verified more rigorously:

• for scores of 0 or 20 it is not possible to detect cognitive-dissonance and so CD_total should always be 0
• as the score tends towards 50%, it should be possible to detect more cognitive-dissonance. A greater confidence should be accorded to the results of the formula in these circumstances.
• finally, the structure of the curriculum assures that in certain key situations, CD_total has either a null or a very high value. These special cases are predictable, and should always be pedagogically explainable.

Figure 7 shows, for each of the 20 possible scores, the distribution of the 175 values of CD_total. First we notice that as the score approaches 10/20 (50%) the dispersion is greater and the values of CD_total tend to be higher (as shown by the correlation curve, a polynomial of fourth degree). The peak is situated between 9 and 10, the scores where highest cognitive-dissonance can be detected. This is consistent with our original hypothesis.

Two outlying points seem particularly important to mention. The first is at (10,0) and corresponds to a situation where the learner has a score of 10/20 but a cognitive-dissonance of 0. The second, situated at (4, 0.52), corresponds to a situation where the low score would normally stop us from determining the mental confusion but where the value of CD_total is remarkably high. How do such situations occur? We give examples illustrating these cases in the next section.

The graph in figure 8 shows that for scores near 10/20 the average total cognitive-dissonance is higher. This corresponds to the hypothesis that a greater part of cognitive-dissonance is detected when the raw score nears 50%. On the other hand, when the score tends to 0 the situation becomes problematic: does the score indicate a lack of knowledge or comprehension? Thus, there is an important conclusion that we reach by analysing this curve: pre-tests should be neither too difficult nor too easy.

Figure 9 shows that the possible pre-test scores are not all equally precise in the detection of cognitive-dissonance. Low to average scores present a high variability which denotes a great richness in results. Often such scores denote very symptomatic tendencies: a learner who has not properly assimilated a key concept or who has not well synthesised the material. It is possible to obtain a score of 10/20 and to have understood everything; the next section shows how this can happen:

4.3 Two Sample Situations

In this section, we present two fictitious cases of students following the course in radiography illustrated by figure 5 and using the simplified curriculum of figure 6. A fictitious dialog between the student and the professor is presented to illustrate what is wrong with the first case.

First case:

Mike has seen a video presentation in which an expert shows how to place the radiographs so that they are in the correct position for the analysis. He hasn't really listened to the whole theory, but he has observed the doctor in action. Before presenting him with an exercise the ITS made him answer a questionnaire of about 20 questions on the subject matter. In it he scored 4/20 with a value of CD_total=0.52 (which corresponds to one point circled on figure 7).

During the exercise (figure 5) he correctly places the images but does not consult the patient history. He seems a little unsure of his manipulations since the troublemaker often makes him change his mind. Clearly he succeeds in some parts of the exercise (placing the images and identifying the features), but he lacks self-confidence and he cannot adequately explain what he is doing (since it is easy to lead him astray).

A conversation with a real professor might look like:

Professor: Mike, we had placed the mammography properly. Why did you turn it?

Mike: HmmÖ the expert in the video did it like that [Mike imitates the expert].

Professor: How did you arrive at your diagnostic of a benign tumour?

Mike: The contour of the tumour was well circumscribed.

Professor: Yes, very good. However it took you three tries to identify the tumour and the tutor gave you part of the solution.

Mike: I could arrive at a solution but the other student [the troublemaker] was bothering me. His solution always seemed better than mine.

Professor: That is because he explains well what he does. Let's see...

[...]

At the pre-test Mike failed all the basic questions (on capabilities c₁,c₂,c₃,c₅,c_6, and c₁₀) but he succeeded in the four synthesis questions. This explains the high result for CD_total and his difficulty in completing the exercise. The conversation with the professor highlights the mental confusion in the learner who believes he knows and tries to show it but is proved wrong. In order to decrease the cognitive-dissonance he finds external reasons (like the troublemaker's interventions) to explain his failures.

Second case:

Janet has missed several important classes since mid-session and has not had the time to read more than half of the book on radiography. Thanks to her great capacity for concentration she has assimilated the basic material very well but would be incapable of performing a diagnostic or of analysing images.

During the pre-test Janet received a score of 10/20, but she succeeded in most of the question related to basic capabilities (capabilities c₁,c₂,c₃,c₅,c_6, and c₁₀). Since she does not know the answers to the questions on diagnostic and radiography analysis she guessed poorly and failed them all (if we had suggested that she skip those questions she would have immediately accepted). Her score of CD_total=0 corresponds to the point (10,0) of the graph which has been circled.

Janet is not ready to attempt the exercise since her knowledge does not permit her to finish it. An expert would arrive at the same conclusion by analysing her score and its implications. Nevertheless the notation used in most institutions of learning concentrates too often on the overall score which represents what the learner knows but not always what he understands. At a grading exam Janet could find herself in a group of students who do not understand the material while her problem is that she does not know it.

5 Conclusion

We have shown in this article that it is possible to quantify the total cognitive-dissonance in a learner. We have also shown that it is possible to give an indication of the cognitive-dissonance for a given capability of the curriculum. The results from this calculation serve to plan external conflicts (between the learner and a troublemaker) in order to make the student aware of internal conflicts, due to cognitive-dissonance, in his model.

In the approach described in this article, co-operation in teaching comes from the efforts of two simulated tutors, one specialised in the transmission of knowledge, the other in a pedagogical aspect of this transmission.

The troublemaker plays the role of a learner following the same course as the real student. The troublemaker strategy allows us to separate the transmission of knowledge from its reinforcement. This allows the student to maintain a high level of confidence in the tutor even though the credibility of the troublemaker may diminish.

Profiting from an external conflict to remedy an internal one seems to us to be a promising research avenue in the framework of social learning.

The troublemaker helps the learner become aware of the incoherence in his ideas and to correct them. Nevertheless it is not possible to do this if we do not have the means for evaluating the internal conflict in the learner. The method presented in this article to calculate cognitive-dissonance explores in more depth what has not been previously broached. What distinguishes it from the formulas proposed by Festinger and his successors is the use of a knowledge structure as basic means (in our case the curriculum). This allows us to complete more complex calculations than the mere comparison of two cognitions. Despite its uneven performance, we believe that the formula presented in this article can be a reliable indicator of the rate of cognitive-dissonance in an individual since using probabilistic methods can predict his behaviour.

Certain questions remain: how can we ensure that the curriculum used is free of contradictions? In science is it correct to consider that there exists only one way to organise the material (a more epistemological question)? How can we ensure that the pre-test is the optimal representation of the curriculum? What role does the indicator of cognitive-dissonance have in the evaluation of the learner?

The method presented in this article is part of a broader process whose goal is to develop, over the long-term, tools to help an expert in a field to adapt a course to a given target public. The characterisation of the target public concerns several aspects including affective (preferences as to teaching style, course presentation mode, etc.) and cognitive (knowledge well acquired, cognitive-dissonance, missing knowledge, etc.) criteria. We are currently working on a program which implements the indices discussed in this paper for the analysis of pre-test results from a group of students.

Acknowledgements

The medical prototype has been developed at the University of Montreal by Daniel Leibu and Hugo Dufort. The radiological expertise was provided by the cognitive science team at McGill University.

This work has been supported by the TeleLearning Network of Centers of Excellence.

References

[Aïmeur and Frasson 96]. Aïmeur, E., and Frasson, C.: "Analyzing a new learning strategy according to different knowledge levels"; (Computers in education), 27(2), (1996). 115-127.

[Aïmeur, Dufort, Leibu, and Frasson 97]. Aïmeur, E., Dufort, H., Leibu, D., and Frasson, C.: "Some justifications for the learning-by-disturbing strategy"; Proc. AI-ED 97, World Conference on Artificial Intelligence and Education, Japan, (1997), 119-126.

[Aïmeur, Dufort, and Frasson 97]. Aïmeur, E., Dufort, H., and Frasson, C.: "Profiting from conflict in intelligent tutoring systems"; Proc. IJCAI-97, Workshop on Collaboration, Cooperation and Conflict in Dialogue Systems, Japan , (1997), 1-7 .

[Baker 93]. Baker, M.J .: "Dialogic learning: negotiation and argumentation as mediating mechanisms"; (invited talk), Proc. World Conference on Artificial Intelligence in Education, Edinburgh, Scotland, (1993), 4-11.

[Behmand Cohen 89]. Behm, J., and Cohen, A.: "Explorations in Cognitive-dissonance"; New York: Wiley. (1989)

[Chan and Baskin 90]. Chan, T.W., & Baskin, A.B.: "Learning companion systems"; C. Frasson and G. Gauthier (Ed.), Intelligent Tutoring Systems : At the crossroads of Artificial Intelligence and Education, chap. 1. (1990)

[Clancey 92]. Clancey, W.J.: "Guidon-Manage Revisited: A Socio-Technical Systems Approach"; Proc. 2nd International Conference of Intelligent Tutoring Systems, Lecture Notes in Computer Science, Berlin: Springer-Verlag, 608, (1992), 21-36.

[Dufort, Aïmeur and Frasson 97]. Dufort, H., Aïmeur, E., and Frasson, C.: "Systèmes tutoriels intelligents: quelques aspects de la stratégie du perturbateur"; Publication départementale #1055, Montreal (Département d'informatique et de recherche opérationnelle, Université de Montréal). (1997)

[Festinger 57]. Festinger, L.: "A theory of cognitive-dissonance"; Standford University Press. (1957)

[Festinger 89]. Festinger, L.: "The Arousal and Reduction of Dissonance in Social Contexts"; S. Schachter & M. Gazzaniga (Ed.), Extending Psychological Frontiers. Selected Works of Leon Festinger. Russell Sage Foundation: New-York, 238-257. (1989)

[Frasson and Gauthier 90]. Frasson, C., and Gauthier, G.: "Intelligent Tutoring Systems : At the crossroads of Artificial Intelligence and Education"; Ablex/ N.J. (1990)

[Frasson and Aïmeur 97]. Frasson, C., and Aïmeur, E: "Lessons Learned from a University-Industry Cooperative Project in Tutoring Systems"; J. Liebowitz (Ed.), (Int. Journal of Failures & Lessons Learned in Information Technology Management), 1(2). (1997)

[Frasson, Mengelle, and Aïmeur 97]. Frasson, C., Mengelle, T., and Aïmeur, E.: "Using Pedagogical Agents In a Multistrategic Intelligent Tutoring System";, Workshop on Pedagogical agents in AI-ED 97, World Conference on Artificial Intelligence and Education, Japan, (1997) 40-47.

[Frasson, Mengelle, Aïmeur and Gouardères 1996]. Frasson, C., Mengelle, T., Aïmeur, E., and Gouardères, G.: "An Actor-based Architecture for Intelligent Tutoring Systems"; Proc. ITS 96 Conference, Third International Conference on Intelligent Tutoring Systems. C. Frasson, G. Gauthier & A. Lesgold (Ed.), Lecture Notes in Computer Sciences, 1086, (1996), 57-65.

[Gagné 84]. Gagné, R.M.: "The conditions of learning"; 4 Ed, Les Éditions HRW Ltée./ Montreal (1984)

[Gilmore and Self 88]. Gilmore, D. and Self, J.: "The application of machine learning to intelligent tutoring systems"; In "Artificial Intelligence and Human Learning, Intelligent computer-assisted instruction", J. Self, (Ed.), Chapman and Hall/ New York, 179-196. (1988)

[Heider 58]. Heider, F.: "The psychology of interpersonal relations"; Wiley/ New York (1958)

[McCalla 90]. McCalla, G.: "The search for adaptability, flexibility, and individualization: approaches to curriculum in intelligent tutoring systems"; Adaptative learning environments: foundations and frontiers, Springer-Verlag/ Berlin, 91-112. (1990)

[Morissette 58]. Morissette, J.O.: "An experimental study of the theory of structural balance"; Human Relations, 11, 239-254. (1958)

[Nkambou, Lefebvre and Gauthier 96]. Nkambou, R., Lefebvre, B., and Gauthier, G.: "A curriculum-based student model for intelligent tutoring systems"; Proc. Fifth International Conference on User Modeling, Kailua-Kona, HW, (1996), 91-98.

[Osgood and Tannenbaum 55]. Osgood, C.E., and Tannenbaum, P.H.: "The principle of congruity in the prediction of attitude change"; (Psychological Review), 62, (1955), 42-55.

[Roschelle 92]. Roschelle, J.: "Learning by collaborating: convergent conceptual change"; In (The journal of the learning sciences), 2(3), Hillsdale, NJ: Lawrence Erlbaum Associates, (1992) 235-276.

[Roschelle and Behrend 89]. Roschelle, J. and Behrend, S.D.: "The construction of shared knowledge in collaborative problem solving"; In C. O'Mally (Ed.), Computer Supported Collaborative Learning. (1989)

[Schank and Jona 91]. Schank, R.C., and Jona, M.Y.: "Empowering the student: new perspectives on the design of teaching systems"; (The journal of learning sciences), 1(1), Hillsdale, NJ: Lawrence Erlbaum Associates, (1991), 7-35.

[Self 88]. Self, J.: "Bypassing the intractable problem of student modelling"; Proc. ITS 88, International Conference of Intelligent Tutoring Systems, Montreal, Canada, (1988), 18-24.

[Slavin 90]. Slavin, R.E.: "Cooperative Learning: Theory, Research, and Practice"; Englewood Cliffs, Prentice Hall/ NJ. (1990)

[Sleeman and Brown 82]. Sleeman, D.H. and Brown, J.S.: "Intelligent Tutoring Systems"; Academic Press/ London. (1982)

[Vallerand 94]. Vallerand, R.J.: "Les fondements de la psychologie sociale"; Gaètan Morin Editeur/ Montreal. (1994)

[Vallerand and Thill 93]. Vallerand, R.J., and Thill, E.E.: "Introduction à la psychologie de la motivation"; Éditions Études Vivantes/Montreal. (1993)

[VanLehn, Jon and Chi 92]. VanLehn, K., Jon, R.M., and Chi, T.H.: "A model of the self-explanation effect"; The journal of the learning sciences, 2(1). Hillsdale, NJ: Lawrence Erlbaum Associates, (19920, 1-59.