Bringing ITS to the Marketplace:
A Successful Experiment in Minimalist Design
Carl Gutwin and Marlene Jones
Alberta Research Council, Calgary, Canada
Patrick Brackett and Kim Massie Adolphe
Gemini Learning Systems Inc., Calgary, Canada
Abstract: Intelligent Tutoring Systems (ITS) have proven to be
effective tools for teaching and training. However, ITSs have not become
common in industrial and organisational settings, in part because their
complexity has proven difficult to manage outside of the research lab.
Minimalist ITSs are an attempt to bridge the gap between research and
practical application; they simplify research techniques while striving to
maintain as much pedagogic intelligence as possible. This paper describes
one such system, SWIFT, that is an example of how a minimalist ITS can be
delivered as a commercial product. We outline some of the issues facing
designers of a minimalist system, and describe the ways that research
techniques have been incorporated into four modules of SWIFT: adaptive
testing, course planning, guidance, and diagnosis.
Despite their many successes [Shute, 1990], intelligent tutoring systems
are underemployed in the world's industries and organisations. Complexity,
size, and lack of tools to assist course authors have all prevented systems
developed in research labs from moving to the commercial world. These
limitations point to the need for techniques and strategies that can adapt
research results to industrial and commercial systems. The idea of minimalist
ITS (e.g. [Brusilovsky et al., 1994], [Winne et al., 1992]) has evolved to
fill that need, and SWIFT is an example of a successful attempt to bring a
minimalist ITS to the marketplace.
SWIFT, shown in Figure 1, is an adaptive learning environment [Jones, 1992]
that has been developed in a joint research venture between the Alberta
Research Council and Gemini Learning Systems. Several of SWIFT's
Figure 1. The SWIFT interface.
elements incorporate techniques from ITS research, including an adaptive
testing facility, a situation guide, a course planner, and a diagnosis module.
SWIFT shows that innovative techniques from research can be successfully
modified for use in the real world. The following sections examine more
closely what it means to minimise an ITS, and then discuss the design of SWIFT
as a minimal system.
2 Minimalist ITS
Many intelligent tutoring systems require computational power that is rare in
real-world training situations. The desire to produce training systems that
behave intelligently, but that are also feasible on a smaller scale, has led
researchers to the idea of minimalist ITS. This area has grown from work on
training shells (e.g. [Major & Reichgelt, 1992] [van Marcke, 1990]) and on
discovery learning environments (e.g. [Shute & Glaser, 1990],
[Elsom-Cook, 1990]). Minimalist ITSs attempt to bridge the gap between
research and practical application by simplifying ITS techniques for use in
computing environments with limited memory, external storage, and processing
speed. Any of the common elements of a typical ITS, such as the domain
representation, the learner model, the pedagogic and domain expertise, the
instructional and delivery planners, or the diagnosis engines, can be
minimised. In addition, minimalist systems often attempt to reduce demands on
course designers, since teachers and industrial trainers often do not have the
time nor the specialised knowledge necessary to build the knowledge structures
used by existing ITSs.
Meeting these needs presents a minimalist designer with several tradeoffs, the
most obvious of which is the balance between power and feasibility. To manage
this tradeoffeffectively, a designer must understand what is lost with each
simplification of a technique, and must look for other ways to bolster the
system's pedagogic capabilities. In most cases, the minimalist approach
implies that a system will gather less information about the student, will
have a less-sophisticated domain representation, and will be able to make only
relatively straightforward inferences from that information. Additional means
must be found to ensure that what is left of the ITS techniques can still be
used to advantage, which often means combining the technique with other more
robust mechanisms that can make up forreduced intelligence. Our experiences
suggest that the minimalist goal involves more than just linear scaling of the
techniques-a simplistic approach will result only in a bad approximation of an
ITS, not a minimalist one.
One approach that we have taken in SWIFT is to involve the learner in the
decision-making processes. Even novices have considerable knowledge about
their own learning needs, and they can monitor and alter the system's decisions
if given the appropriate support. Other minimalist approaches take strategies
from traditional computer-based training (CBT) and enhance them with ITS
techniques. For example, CBT is often based upon learning by presentation and
questioning. By providing individualised feedback on a learner's answers, an
instructional system can add the individualisation of ITS to the simplicity of
Course designers also play an important part in the successful application of
minimalist ITS. Authoring must be made accessible so that domain experts can
readily construct new training materials; they must also be able to understand
the strengths of the tutoring system in order to maximise its capabilities.
KAFITS [Murray & Woolf, 1992] and COCA [Major, 1994] are two examples of
systems that concentrate on providing tools for authors; SWIFT also provides
extensive support for course design [Massie, 1994].
3 Minimalist Design in SWIFT
The following sections describe the approaches that we have used to make the
most of the resources available to our system. In general, our strategies have
taken three paths: first, we have found ways to minimise ITS techniques without
compromising too much oftheir power; second, we have found additional
mechanisms to make our solutions more robust; and third, we have taken
advantage ofthe abilities oflearners and our knowledge of the eventual user
3.1 Knowledge Representation
A defining feature of an ITS is a semantic representation ofthe instructional
domain, where concepts are encoded in data structures that allow the system to
reason about the course. A minimalist ITS must also employ semantic
representation, for an understanding ofthe concepts in the domain is the basis
of much of a system's intelligent behaviour. However, the detail and
sophistication of the representation can vary. In SWIFT, we have implemented a
representation scheme that allows us to reason about the domain, but does not
contain as much
detail about specific concepts as might be found in a full ITS.
SWIFT courses are stored in a hierarchical structure that divides the
instructional material into smaller and smaller pieces, much as a book does
with chapters, sections, and subsections. A course has three levels: the first
contains a set of topics, which are divided at the second level into sets of
modules, which are divided at the third level into concepts. A semantic
representation of the course also allows the specification ofdependencies
between concepts. The current version of SWIFT allows for prerequisite and
sequence links between individual concept objects.
3.2 Adaptive Testing
ITSs gather information about a learner's progress by observing them as they
interact with the learning environment. Many minimalist systems use exercises,
quizzes, and exams as the setting for these observations, since the range of
possible inference about the learner can be more easily constrained. Since many
organisations (corporate and otherwise) also require that a training system
provide concrete records of progress, we have chosen to use formative and
summative testing as our means for observing the learner in SWIFT.
One of the problems with traditional exams is that they are offixed length; a
learner must complete a long series of questions in order for the system to
determine how well they know a subject. This characteristic can cause
frustration for both novices and experts, who may know after a few questions
that the subject matter is either bewildering or trivial. Aside from giving the
learner greater control over exams-in that they are never forced to take a
test-our primary strategy for tackling the problem of fixed-length exams is
adaptive testing. Adaptive testing allows exams to be significantly shorter
than traditional tests, without losing any predictive power about a learner's
mastery of the material. The approach that is implemented in SWIFT is based on
the work of [Welch & Frick, 1993]. The algorithm uses Bayes, theorem to
estimate the probability that the learner is a master or non-master of the
material after each test question is answered. In SWIFT, novices (non-masters)
and experts (masters) can be determined in as few as five questions.
3.3 Instructional Planning
Instructional planning in SWIFT is based on two information sources: the
results of an adaptive pretest, and the learner's own choice of one or more
instructional goals. Each goal specifies which topics and modules of the
course are to be included in the learner's path; performance on the pretest
then indicates whether concepts within those sections are already known and
need not be included. Our approach to instructional planning is effective,
but is relatively simple compared to some ITSs (e.g. [Brecht, 1990]) because
of SWIFT's less-sophisticated domain representation. Since our simpler
approach weakens SWIFT's planning to a degree, we have found other ways of
ensuring that appropriate instruction is always available to the learner.
Since we knew that the target population for SWIFT is composed largely of
learners that are cooperative and motivated, we were able to view instructional
planning as a human-computer problem rather than just a computational one. One
of the ways we involve the learner is by providing tools that allow them to
monitor their path through the course, and to take control if desired. Figure 2
shows a concept map in SWIFT, one of several displays that explicitly lay out
the content and dependencies of a course, and allow the learner to make
informed decisions about what to learn next. This approach improves
instructional planning by making use of the
Figure 2. A SWIFT Concept Map.
knowledge of both parties: learners can improve upon or customise the system's
course plan ifthey wish; the recommended path, which is adequate in most cases,
provides support for learners who do not wish to venture out on their own.
Diagnosis modules attempt to understand problems and misconceptions in a
student's knowledge ofthe domain (e.g. [Johnson & Soloway, 1985],
[McCalla & Greer, 1990]). Although any student action may be considered,
diagnosis is commonly applied to a learner's answers to test or exercise
questions. Diagnosis entails drawing conclusions about the learner's knowledge
based on features in their answers; good diagnosis allows systems to provide
appropriate feedback and remediation as well as simple indications of whether
an answer is right or Wrong.
Diagnosis can require significant inferencing power and domain knowledge,
which are not the strengths of minimalist systems. An alternative to a fully
knowledge-based approach is to detail a number ofcategories, or cases, of
typical errors and misconceptions. Using a case-based approach transforms the
inference problem to one of classification, but effective classification can
also be difficult to achieve. One problem occurs in specifying the answers
that belong to a particular class. The obvious method is to encode every
answer. However, this technique implies that any variation of an answer, even
those that do not change its essential parts, must also be included. This can
be a daunting task for any but the most trivial of exercises.
Our approach to this problem allows a course designer to concentrate on the
qualitative differences in the possible answers to a question, rather than on
syntactic variations. Our case-based diagnosis subsystem uses regular
expressions, constructs that allow a designer to specify a large number of
possible variations with a single answer pattern. The system can examine and
evaluate any short textual answers for which cases have been designed. The
course designer specifies patterns for classes of correct and incorrect
answers, and can annotate each class with appropriate feedback and remediation
This strategy still requires that the course designer understand the kinds of
difficulties that learners can have in a particular area, and how each problem
can be manifested in answers to questions. However, we have provided a
framework for structuring and using that pedagogic knowledge that is both
powerful and efficient enough to be used in a minimal system.
3.5 Situation Recognition and Guidance
SWIFT has more and more become a learner-controlled system, both by design and
by necessity. In a self-directed environment, the task ofthe intelligent
tutoring system shifts from tutoring and control to guidance and support. We
have been forced to find and implement mechanisms for supporting learners as
they explore the system on their own.
We have developed a subsystem within SWIFT that can provide guidance on
pedagogic issues according to the specific situation that the learner is in,
and can also encourage the learner to initiate certain learning behaviours.
Many strategies exist for assisting self-directed learning that promote
metacognition and more effective learning behaviour (e.g.
[Derry & Murphy, 1986], [Derry, 1992], [Pressley et al., 1989],
[Shuell, 1992], [Winne, 1992]). Examples of effective learning behaviour
include positive self-talk, note-taking or highlighting, summation, imagery,
question-generation, and review oflearning objectives.
SWIFT's guide watches system events and monitors a learner's location, history,
and current knowledge. When particular kinds of situations occur, the guide can
decide to deliver advice to the learner. For example, if a student turned their
attention to a new section of course material, the guide might suggest that
they test their knowledge of the current section before going on. The guide is
implemented as a rule-based system, and the above example would involve a rule
such as: 'if the learner has not demonstrated mastery in the concepts of the
current module, and the learner requests a move to a new module, the system
will suggest that the learner take a module test for the current module.' The
guide's advice is presented in a popup dialogue box, such as the one shown in
The rule-based guidance system provides SWIFT with a generalised architecture
for presenting useful information. We are able to give the learner pedagogic
guidance in a wide variety of situations, but the
Figure 3: A SWIFT Guidance Window
architecture can also be used to give information about any situation, such as
tips on using SWIFT to its full capacity.
4 Current Success and Plans for Further Work
SWIFT has now been released as a commercial product, and a number of
organisations are producing courses in domains as varied as air traffic
control, Canadian history, high school physics, and football. Designers have
so far found course development to be straightforward, but we are planning for
a graphical authoring environment to further support the authoring process.
SWIFT has been evaluated through formal usability studies involving
representative users from industrial settings and from secondary and
post-secondary education institutions. The usability testing has validated
many of our design decisions, but has also caused us to refine some parts of
the system. For example, some users felt that the guide offered advice in to
many situations; we have since tuned the guide's rules to reduce repetitive
or spurious advice.
We are currently investigating other techniques from ITS research that may be
appropriate for implementation in a minimalist system, as well as planning
commercial improvements such as more sophisticated multimedia and hypertext
support. Some of the possibilities for the next version of SWIFT are:
- Granularity-based diagnosis [McCalla & Greer, 1994];
- Collaborative learning tools, such as support for awareness of other
learners (e.g. [Ayala & Yano, 1994]);
- Improved student modeling based on recent techniques of modeling learners
based on test results [Shute, l994].
Our experiences have shown us that minimalist thinking involves more than
taking existing techniques and scaling them down, and that addition a
mechanisms must often be found to ensure robust and rewarding interaction with
a minimalist ITS. The examples ofour design efforts in SWIFT suggest that
minimalist ITS can be successfully constructed within the constraints of the
SWIFT is available from Gemini Learning Systems, Inc. and from its
Several people have contributed to the design of SWIFT and played a part in
the ideas presented here. Thanks to Stuart Williams, Ruby Loo, Joseph Poon,
Julia Driver, and Jim Tubman. Special thanks to Jim Tubman and Pam Hirtle for
assistance in long-distance preparation ofthe final draft.
[Ayala & Yano, 1994]
Ayala, G. and Yano, Y. (1994) Design Issues in a Collaborative Learning
Environment for Japanese Language Patterns. Educational Multimedia and
Hypermedia, AACE, 67-72.
Brecht (Wasson), B. (1990) Determining the Focus of Instruction: Content
Planning for Intelligent Tutoring Systems. Unpublished doctoral dissertation,
University of Saskatchewan.
[Brusilovsky et al., 1994]
Brusilovsky, P., van Marcke, K., Murray, T., Major, N. and Vassileva, J. (1994)
Minimalist ITS. Panel Discussion, Educational Multimedia and Hypermedia, AACE.
[Derry & Murphy, 1986]
Derry, S., and Murphy, D.A., (1986), Designing Systems that Train Learning
Ability: From Theory to Practice. Review of.Educational Research, 56(1), 1-39.
Derry, S. (1992) Metacognitive Models of Learning and Instructional System
Design. In Adaptive Learning Environments Foundations and Frontiers,
M. Jones and P. Winne ed. Springer-Verlag. 257-286.
Elsom-Cook, M. (1990) Guided Discovery Tutoring. London: Paul Chapman
[Johnson & Soloway, 1985]
Johnson, W. L. and Soloway, E. (1985) PROUST: An Automatic Debugger for Pascal
Programs. Byte, 10 (4), 179-190.
Jones, M. (1992) Introduction. In Adaptive Learning Environments Foundations
and Frontiers, M. Jones and P. Winne ed., Springer-Verlag, 1-10.
Major, N. (1994) Evaluating COCA - What do teachers think? Educational
Multimedia and Hypermedia, AACE, 361-366.
[Major & Reichgelt, 1992]
Major, N. and Reichgelt, H. (1992) COCA: A Shell for Intelligent Tutoring Systems. Intelligent Tutoring Systems, Springer-Verlag, 523-530.
Massie Adolphe, K. (1994) AI and Education. Canadian AI Magayine, 34 (1), 4-8.
[McCalla & Greer, 1990]
McCalla, G. I. and Greer, J. E. (1990) SCENT-3: An architecture for
intelligent advising in problem-solving domains. In Intelligent Tutoring
Systems At the Crossroads of.Artificial Intelligence and Education,
C. Frasson and G. Gauthier eds. Norwood, NJ:Ablex, 140-161
[McCalla & Greer, 1994]
McCalla, G.I, and Greer, J.E. (1994), Granularity-Based Reasoning and Belief
Revision in Student Models, In Student Modelling The Key to Individualized
Knowledge-Based Instruction, J. Greer and G. McCalla eds,
[Murray & Woolf, 1992]
Murray, T. and Woolf, B. (1992) Tools for Teacher Part\icipation in ITS Design.
Intelligent Tutoring Systems, Springer-Verlag, 593-600.
[Pressley et al., 1989]
Pressley, M., Johnson, C., Symons, S., McGoldrick, J., Kurita, J., (1989),
Strategies That Improve Children's Memory and Comprehension of Text.
The Elementary School Journal, 90(1), 3-32.
Shuell, T. J. (1992) Designing Instructional Computing Systems for Meaningful
Learning. In Adaptive Learning Environments Foundations and Frontiers,
M. Jones and P. Winne ed., Berlin:Springer-Verlag, 19-54.
Shute, V. (1990) Rose Garden Promises of Intelligent Tutoring Systems: Blossom
or Thorn?, Space Operations, Applications and Research (SOAR) Symposium,
June 1990, Albuquerque, NM.
Shute, V. (1994) Regarding the I in ITS: Student Modelling. In Educational
Multimedia and Hypermedia, AACE, 50-57.
[Shute & Glaser, 1990]
Shute, V. J. and Glaser, R. (1990) A Large-scale Evaluation of an Intelligent
Discovery World: Smithtown. Interactive Learning Environments, 1(1), 51-77.
[van Marcke, 1990]
van Marcke, K. (1990) ``A Generic Tutoring Environment.', The European
Conference on Artifical Intelligence, 655-660.
[Welch & Frick, 1993]
Welch, R.E., and Frick, T.W. (1993), Computerized-Adaptive Testing in
Instructional Settings. Educational Technology Research and Development,
Winne, P. (1992) State-of-the-Art Instructional Computing Systems that Afford
Instruction and Bootstrap Research. In Adaptive Learning Environments
Foundations and Frontiers, M. Jones and P. Winne ed.,
[Winne et al., 1992]
Winne, P., Butler, D., McGinn, M., Sugarman, J., Jones, M., Mark, M., and
Field, D. (1992) STUDY: A Tool for Authoring Adaptive Learning Environments
and for Advancing Instructional Research. In Appendix to Proceedings of.ITS,92.