Applications of MIRACLE: Working With Dynamic Visual Information

Abstract: Systems supporting new forms of communication and archiving of dynamic visual information have a range of potential applications, some of which are described in this paper on a conceptual basis. We present a visual language for dynamic (historic) maps, applications of pictorial lexicons, concepts for interactive support systems for assembly and repair, and a platform for abstract movies.

Keywords: visual languages, visual communication, interactive maps, computer-supported communication, information archiving, information retrieval, multimedia, dynamic maps, abstract movies

Categories: A.1, H.3.7, H.4.3, H.5.1, J.4, J.5

1 Introduction

In [MSC03] we gave an extensive survey of various issues of visual communication with a particular focus on efforts to develop computer-based platforms supporting dynamic visual languages for human-to-human communication. As one major conclusion we found that in the past dynamics have not been investigated comprehensively on a structural level. Also, "natural animation" as a means of representing dynamical processes has not been exploited fully enough in the context of computer-supported knowledge transmission and archiving. In this paper we show how the availability of computer-based tools supporting visual communication could greatly improve information sharing and archiving in a number of specific applications.

2 Dynamic Maps

2.1 The visual language of maps

Maps are a powerful tool to organize, store, and communicate concepts in a visual manner. Indeed, for many types of information it is obvious to be represented in the form of maps, in particular if they are based on spatial data such as geographic information (see [Ovi96], [AA99], [BS01], [Pet95]). Therefore it is clear that geographic information from traditional cartography to computer-supported Geographic Information Systems (GIS) heavily rely on map based representation.

Other kinds of maps, however, have become increasingly popular recently as powerful tools to organize and share thoughts and concepts. Mind Maps(®) as conceived by Tony Buzan ([BB96], figure 1 shows an example) is one of the most popular examples, but a range of other kinds of "concept maps" have been developed. In today's computer based environment, the creation, sharing and storage of such information is usually supported by appropriate software. In human-computer interaction, information visualization also relies on sophisticated visual metaphors including "information landscapes".

Most times, information is displayed in a networked and hierarchical way, like in the "Knowledge Net" of an electronic encyclopaedia shown in figure 2 which automatically groups together the most relevant articles around a current entry positioned in the center.

Although interfaces to such systems may allow dynamic interaction with the information represented (such as expanding/collapsing information structures, flying through 3D information landscapes, rotating information networks etc.) as long as the information itself is static, so naturally is its depiction. But if a temporal aspect is included, a further dimension has to be coped with. Visual representation of spatio-temporal information poses new challenges and historic maps are an example of such applications.

One of the main issues in such systems is the question how to represent the time factor. There are basically two ways: one can store entire "map slices" as discreet snapshots for each moment that contains information differing from any other slice. The other way is to store only the changes (and, of course, an initial state). In the case of historic maps, the first approach would result in a number of full maps, whereas the second one would for instance only store the polygons of the changed country boundaries (including erasure of old ones), unfolding to the full historic development over time. Technically, the first approach could be covered by bitmaps whereas the second one would be better treated by vector graphics.

As a static historic map taken out of a paper-based book, figure 3 shows the system of alliances in Europe shortly before World War I ([dtv91]). Overlaid over the actual geographic map is a symbolic representation of the alliances, in the form of circles and half-circles in varying colors and styles. There are also some specific symbols (such as flames denoting conflicts) and arrows (denoting political influence and/or territorial expansion).

The image expressively demonstrates the power of visual representation of complex situations. Intuitive to a high degree, the reader just has to read a brief legend and be familiar with a small number of abbreviations to understand the picture.

It seams feasible to describe historic events on an abstract level with such visual representations in mind. Historians should work together with computer scientists to specify a language as a basis for such descriptions and to develop corresponding software systems that would allow to author dynamic representations of historic developments in a comfortable way. Such a language could serve as a powerful medium for communication about history and for storing corresponding educational material. MIRACLE could help greatly in these aspects.

Let us go into more detail concerning a visual language for the description of historic events and political situations. Historic atlases use a range of visual means to depict various aspects of territorial developments and political circumstances (change of boundaries, strategic alliances etc). A number of waves of computer-assisted instruction since the 1960s have brought a range of computer-based variants of historic maps, with or without interactive and dynamic features. The problem with such systems always was the high costs of producing the content. The boundary of every country had to be draw manually and the areas shaded properly, the same with dynamic effects and animations to show political developments. A specification system would help greatly in that respect.

Therefore we propose a system for dynamic history visualization called Dynavish (DYNamic Abstract VISualisation of History) which provides the framework to specify a language for the visual description of historical processes, involving either concrete representations of territory (in form of countries shown on maps) or abstract representations of political entities (countries, alliances, pacts etc.). The basis of the system would be a database storing the following objects, potentially with varying values following the temporal axis:

• Places (mainly cities): Name and geographic coordinates;
• Persons: Name, dates of birth and death, country; role(s) (out of a large predefined table);
• Countries/territories: name and form of government (out of a predefined set of possible values) per period of time; geographic coordinates for specific moments (or periods) of time; data would preferably be available in numeric form (coordinates), as opposed to plain graphical form (such as scanned drawings); capital (a reference to a place); includes/is-included relation to other entries of the type country (to allow provinces to be related to states etc.);

Based on these data, ideally the system would be able to draw maps of desired areas for any specified period of time, automatically shaded in an appropriate and appealing way. The user would be able to customize the visualisation, for instance to give countries with the same (or similar) systems of government the same color.

2.1.1 Dynamic historic visualisation

So far the system described would be nothing more than a comfortable computer-based historic atlas. Only with the introduction of dynamic features Dynavish unfolds its full innovative power. To this aim Dynavish features a script language called Hiscript. It is stored as a plain text script that can be edited directly in any plain text editor, but Dynavish will also provide a WYSIWYG ("what you see is what you get") editor for the comfortable creation and maintenance of Hiscripts.

A Hiscript describes a historic process referring to the data stored in the system described in the previous section. These data are connected by a human author who structures and sequences them with the knowledge of a historian. Dynavish offers two dedicated modules to allow the definition of

• relations between objects stored in the database in a specific moment of time, and
• processes involving both objects and relations.

The description of relations (and their development) between entities of human organization is very important for understanding history. Such relations can have a range of types such as political, religious, cultural etc. Dynavish provides a module to define such relations in a comfortable way. The author selects (groups of) countries (or, on a more general and abstract level, political entities) and chooses a type of relation, plus the way the system will depict it (straight lines or (half) circles, type of lines, color etc.).

If more than one relation is defined for an entity at a time, the system proposes an appropriate way to simultaneously show them all. The system automatically takes care of a proper visualisation of the relations based on these specifications.

Processes are defined on a higher level, referring to both objects and relations. To define a historical process the user would have to define start and end points by selecting static maps or relations from the database for the corresponding moments of time. In doing so, the temporal framework of the process would be specified. Then the author would have to decide which intermediate states should be integrated in the process (given there are items between the start end points available in the database) by way of interpolation done by the system (fading away objects no longer active, and fading in newly appearing ones; smoothly showing expansion and shrinking of countries etc). Figure 4 shows a sequence of 6 maps (again from a printed source) that Dynavish ideally should be able to integrate to one dynamically floating presentation.

In Hiscripts, the display of arbitrary multimedia elements (tables, charts, written text, audio, video, animations etc.) can be synchronized with the animation of the processes generated by the system itself (defined by the script) to deepen the user's understanding with illustrations and further written or spoken explanations.

Alliances are represented as lines connecting countries. Depending on the geographic situation the system depicts this in the form of a circle, an arc, or a straight line going through the centres of the countries, with the colour automatically selected if not defined by the author (colour can have "political significance" as we know).

To enhance consistence, various aspects (countries and political, economic or religious information) can be represented by colour or shading effects over a period of time. "Playing back" a sequence of historical events in a specific geographic area would result in a sort of a highly abstract movie which would quite intuitively show the developments. The images could be accompanied be acoustic information, most probably spoken text explaining further details of the shown developments. The user would be able to stop the playback at any time to investigate aspects directly by clicking on visual items. The system would then present details for the item as stored on the database for the specific time. Note that both spoken as well as written information can and should of course be made available on a multi-lingual basis.

2.1.2 Visualizing social and organizational relations and processes

As a side effect the Hiscript module could - with a few modifications - also be used for visualizing relationships between humans on a smaller scale, say in the framework of a company, community or even a family. In that way a window for numerous further applications would be opened.

3 Instruction

3.1 eLearning

Visual information can significantly increase the learning rate in many situations, and quite often proper instruction would actually be unfeasible without images. In traditional instruction, illustrations in printed books have fulfilled that task. Today, with the wide availability of electronic media, multimedia elements are commonplace in modern instructional settings, allowing for much more intuitive and direct learning experiences with the help of dynamic and interactive components. Basing such systems on visual components to an even stronger degree by using a visual language as proposed in MIRACLE could further enhance the value and efficiency of instructional systems.

The high costs of producing high-quality instructional material have always been a major problem in eLearning efforts. In particular, a visual communication platform such as MIRACLE needs to be based on a large image collection serving as a visual dictionary. Indeed pictorial dictionaries such as the ones produced by Oxford-Duden [OD95] offer a reference between an image and its proper verbal correspondence. They can be used to find the proper words for rarely named objects (such as specific types of sails, for instance) and - if they are multilingual - to find simple one-to-one translations of words with their images acting as a universally understood "mediator". Of course this only works in the realm of the physical world because abstract concepts, feelings etc. cannot easily be depicted visually (at least not on a generally agreed basis).

Pictorial dictionaries can be used to support the production of courseware in a semi-automatic way. For instance, in an instructional unit on large sailing ships, figure 5 could be used as a starting point to explain all the different types of sails, one by one.

The basic words and their references to the graphics (via the index numbers) are already available. These just need to be expanded with detailed explanations that can come in any type of multimedia. Sequenced appropriately, the result would be a simple instructional unit.

The scenes shown in figure 6 are of a different quality. Here, the objects are basically the same in all five panels: a swimmer (whether male or female is an irrelevant detail). What makes the difference here is what they are doing, i.e. the type of swimming. Of course in this case it would be much more natural to show the movements directly, in the form of animations or video clips, thus it would be much better handled on a computer-based platform than on paper.

Figure 7 shows yet another "visual setting". Here, die overall situation (a hotel bar) to a large degree defines the meaning of the individual visual items and thus we learn that the proper term for the man behind the bar (label no. 62).

The precise name of an object depends on its context and images can be useful to provide such backgrounds. The relations between textual and pictorial information about the items (represented on paper by the numbered labels) can be exploited by computer systems in a number if ways.

In a computer-supported environment for visual communication as outlined in the first MIRACLE paper [MSC03] users could make references to specific items in pictorial dictionaries in case they need to refer to unusual terms or very specific objects. In that way, translation between languages is made available as a side-effect, assuming the pictorial dictionary has a multi-lingual text basis. Thus in MIRACLE a seamless switch between instruction, communication, and knowledge retrieval would be possible in a visual environment.

3.2 Instructing Pre-literate Children

We now focus on children, who can already talk fluently and are capable of handling simple computer games or such, but cannot fully read and write yet. In Middle Europe this group would typically comprise the children aged 4-7, i.e. almost 10% of the total population! In addition to the obvious and most essential "learning by doing" way, children in this age group usually learn by watching and listening to adults and (in the so-called "developed world") by watching television.

There are many situations when other alternatives like a subset of MIRACLE mixed with multimedia clips would be very convenient. Let us examine the particular instance of games. There are many games (card games, board games, ...) that are available for children of the mentioned age group and which are well-liked by the children, yet the explanation of the rules of the games comes as printed booklet, i.e. requires an adult 'interpreter'. It should be clear that packaging the games with a multimedia CD that shows how the game is played using examples in connection with obvious symbols (like wagging a finger for "this is a no-no" or a thumbs up for "this is very good") would make lots of sense.

A combination with a WWW server may still be more powerful, since by dropping 'question icons' at some places not only more explanation can be given (as would also be possible with CDs) but the game manufacturer obtains valuable fee-back on where children have difficulties to understand the game. That such a networked environment can also be used to form a community of players, possibly communicating using MIRACLE is another aspect that has received little attention so far although computer game industry has introduced online games and associated player communities long ago.

MIRACLE could support children not only in learning how to play games but also in learning important things for the "serious life" (like tying shoe-laces) or how to assemble objects correctly out of complex construction kits. These issues are going to be addressed in a forthcoming paper: A Child's CanDo Assistant; Lennon, J. Maurer, H., JUCS vol.9, no.9 (September 2003), to appear.

3.3 First Aid

First aid requires precise and quick action. Therefore any first aid instruction has to be confined to the simplest and briefest possible text and clear illustrations. Figure 8 shows a page of a book ([Gre92]) on first aid for children which fulfils that requirement.

Because emergencies cannot be predicted, a computer-based first aid support must be available "anytime" and "anywhere", based on some sort of mobile device. PDAs or UMTS mobile phones could be appropriate platforms for such systems. Simple graphic and textual instructions could be made available from a dedicated server. In addition to the general information coming from the server, a human expert could provide precise and individual instructions on the basis of spoken information given by the rescuer and of images captured with a built-in camera. The instructions provided by the person "behind" the server could again be individual spoken instructions or playing back pre-produced sequences of animation or video.

The application of first aid as described here is actually similar to the one described in the following section and it is clear that it would likewise heavily benefit from the more advanced mobile devices described there.

3.4 Electronic Support for Assembly and Repair

Current developments in Augmented Reality and Wearable Computers (of the many resources see for instance [Ge00], [DV01], [Fei02], [MO03]) indicate that in the future computers will be able to provide proper support in situations requiring specific assistance by giving feedback to an actual and individual setting and context.

Among the applications most appropriate for and demanding such assistance are all situations involving some very specific physical arrangement of items, such as repair and assembly.

On early generation systems users would give input to the system by manoeuvring through a hierarchical menu structure or through direct search functions. More advanced future systems would allow visual input through a built-in camera. Image analysis techniques would allow to automatically recognize certain circumstances shown in the image, such as broken items or configurations that need specific maintenance. The system would generate visual output on mobile devices (such as PDAs or various sorts of head-mounted displays). The presented content could be a combination of video, animation, and "visual statements" composed of components of a visual language as proposed in MIRACLE.

Although currently it seems to be a long road in that direction the, ultimate "mobile assistance system" (see also the eAssistent outlined in [MO03]) would be as easy to be used as pointing a small device (including a camera) towards some item or "scene" that requires some action and watching what assistance the systems would offer by visual explanation. The assistance would not be generic but would be centred around specific support for an individual situation, intuitive and easy to understand.

Note that there are a number of very specific variants of such general applications, such as first aid (see previous section) and telemedicine, instruction for children on games (see above) or the assembly of construction kits, and various forms of repair and maintenance tasks, be it in private or professional settings (automobiles, bikes, machinery etc.). Furniture manufacturers and retailers could bundle their products with corresponding electronic assembly assistance.

3.5 Mobile Communication: Directory, Routing and Ordering Services

The fast development of mobile communication will open another field of applications for visual communication in the near future on the basis of widely available technology. Possible applications range from hotel and travel booking to information systems offering train or flight schedules, ticket ordering and many others. Many of these fields will see simple realizations soon, based on UMTS technology. However, the development of sophisticated visual communication tools for small mobile devices (including computer driven control centres found in automobiles) will remain a challenge for some time. In particular, new forms of visual metaphors will be required to fully exploit their limited displays and it can be assumed that dynamic elements will play an essential role in that respect.

4 Communicating in Groups With Special Needs

Computer-supported visual communication platforms could significantly improve sharing information within certain groups that have special needs, as well as between members of such groups and the "rest of the world". We give a few examples.

4.1 Aiding Hearing Impairments

The natural language of the deaf is sign language. With specific grammars and large vocabularies, sign languages are about as difficult to learn as spoken languages and they come in many local variants quite similar to spoken languages. Therefore deaf people from different regions can only communicate using a specific sign language of some international significance in the sense of a foreign language. Today, American Sign Language (ASL) serves a similar global purpose as second language for the deaf as does English for the hearing. Due to its strong relation to spoken language, written text is not an appropriate means of communication for people born deaf. In this regard MIRACLE can play a big role for the community of deaf people, serving as a cross-cultural platform, both among deaf communities using different sign languages as well as between the deaf and hearing people.

Computer-supported systems doing real-time translation between sign language and spoken language (in both directions) are being developed, but a system like MIRACLE would offer a key advantage to such systems: being of a more general kind, it would be useful also to the average population.

Thus, MIRACLE could be of tremendous help for deaf people, particularly if every attempt is made to take into account in the design of MIRACLE typical elements of sign languages: this is one of the many reasons why the development of MIRACLE will closely look at all aspects of gesture languages (for more details in this aspect see [MSC03].

4.2 Against Illiteracy

There are a number of substantial groups of persons that are not capable of writing and reading well. Reasons for this can be

1. Age: children under a certain age just cannot read and write; this group is covered in more detail above.
2. Dyslexia: Various forms of mental deficiency, be it by birth or caused by an accident can seriously limit the ability to read and write.
3. Deafness: As mentioned above, people born deaf have substantial difficulty to handle written text because of its strong relationship to spoken language.
4. Environment: A large proportion of persons in third world countries cannot read and write because they have no opportunity to learn this skill.

For persons in all these groups, using symbols for generating or using information would offer a significant relief or assistance, and even a small subset of MIRACLE could prove very helpful. Both communication within the mentioned groups and between those groups and the average majority of population would strongly benefit from such a system. It has to be noted however that in case (d), one of the basic assumptions that computers are universally available to all may not be realistic in the near future. Hence, static versions of MIRACLE (i.e. printed visual languages that might show some similarity to Blissymbolics, see [Bli84]) may be more likely to become popular on a short-term perspective.

5 Abstract Movies

In the context of movies, as with textual documents, the word "abstract" has a whole spectrum of meanings. It can mean obtuse, complex, indefinite, subtle, abbreviated, etc.

In the context of hypermedia systems a single movie may also have any, or all, of these representations associated with it. In the first MIRACLE paper [MSC03] and the first MUSLI paper [ML94] we have described how elements of a movie, both static and dynamic, may have multiple representations. These alternatives may be stored as linked resource files. By invoking a "preferences" option, users may define how they wish the alternatives to be presented. For example, we showed how a movie might provide the following views:

1. Symbolic - condensed: abbreviated forms
2. Symbolic - expanded: any of a multiple range of more explicit forms, including attributes
3. Iconic
4. Pictorial and/or graphical
5. Written textual details
6. Digitised sound/speech

We have produced prototypes that let users play and replay movies displaying any of the above forms - or a selection of them simultaneously. This level of choice makes it possible to tailor MUSLI documents to suit users' individual differences, preferences, and learning/ reading styles.

5.1 Preferences

The interplay between authors and reader's views of any document is a particularly interesting, albeit complex, one. For example, no matter how precise an author may try to make a document, readers' backgrounds will almost inevitably colour their interpretations. Many people (particularly teachers and dictators!) have viewed this as a weakness to be overcome at all cost. However, we believe that this very weakness can, and should, be turned into a strength. This is particularly true with movies. How many times have you enjoyed reading a book only to be disappointed with the movie that was based on it? The problem is frequently that the movie contradicts assumptions that we formed while reading. Often it is not abstract enough to comply with visualizations formed by our imaginations (conscious or unconscious).

5.2 User-directed Representations

Ideally, the movie should be presented to us in a form that we feel most comfortable with at the time we view it. This assumes that as we learn from the experience we may then want a different version of the movie at the next viewing. In other words, users should be able to control the presentation to reflect their particular requirements at any one point in time.

We prototyped a version of Snow White that demonstrated how this could be achieved - at least on a few levels [Ho00]. We chose to look at levels of abstraction. As can be seen in figure 9 users can run and re-run the movie with various filters selected from a predefined list.

Figures 10 (a, b, c) show the same scene but with (a) iconic representations of Snow White and the dwarves, (b) a Disney representation, and (c) an abstract (symbolic) filter.

In addition to this, all the major characters in the movie had multiple representations as illustrated in the scrollable list in figure 11.

What we felt was most impressive was the interest that all our user testers have expressed in the transition from icons to abstract symbols. This was particularly true for the sections of the movie with high emotional content such as fear, see figure 12.

When the whole movie was viewed with representations such as those captured in figure 12 (b) or (c) the interplay of emotions was emphasised in dramatic fashion.

Levels of abstraction are especially important in learning environments, enabling different learning styles to be supported more easily. This also gives teachers the chance to adjust the initial level of abstraction, or concreteness, to better suit individuals.

An additional point that we feel is of importance is that it also helps users to retain their power of imagination - thereby unifying the advantages of word and picture.

Yet another way of viewing abstract movies that may suit youngsters in this multi-stream world is, as shown in figure 13, to present alternative views simultaneously. Since documents might contain deliberate ambiguous meanings these could be a fascinating way of comprehending the author's message.

5.3 Artistic Representations of Movie Clips

To the above list we now want to add artistic variants of real-life video clips. Consider the following illustrative example:

A scene from the novel The Mindcaller is filmed where the actress playing the part of Aroha finds an obsidian carving and sees a "flash" of her Nan's face. The film is then digitised into separate layers: background, actor(s), and soundtrack. The background (landscape) could then be transformed, into, say, an impressionistic style. The actor could be rendered in surrealistic form.

Nan's face could be transformed into a post-modern art form that focuses on enlargement and fracturing of various facial parts. The transformed movie could then show those facial reformations, as emotions are displayed, in a very unique way.

Possible options:

1. Art style or art movement
   Examples:
   Classical - neo classical - contemporary classical
   Surrealism - pop surrealism
   Sublime - neo sublime - horror sublime - sticky sublime
   Realism - neo realism - neurotic realism
   Modernism - fauvism - abstraction - minimalism - conceptual art - feminist art - expressionism
   Post Moderism - retro feminism - post minimalism - postmodern sublime
2. Genres within each movement
   Examples:
   Landscape
   History Painting
   Portraiture
   Still Life
3. Current genres that could be researched include:
   - quotidian, graffiti, banal, process art, etc.
4. Artists with distinctive styles include: Leonore Fini, Dora Carrington, Frida Kahlo (for Surrealism) and Matisse, Kirchner, Gunter (for Modernism - Expressionism).

Incorporating features such as this, the MIRACLE prototype system will facilitate:

• The exploration of transformation rules for the conversion of video clips (of drama actions) into alternative artistic genres.
• The semi-automatic production of personalized, multi-layered painterly video clips using generic transformation rules expressed in the works of impressionistic painters such as Francis Marno (modern) - or Monet, van Gogh, etc. (historical).
• The use of a range of pull-down menu options. For example, options for the Medium (oils, acrylic, watercolour, gouache), Palette (colours, colour mixes), and Equipment (brushes, rags, paint rubbers, sponges, scrapers) may be used.

Hence, with MIRACLE, users will be able to transform their own video clips into a range of artistic variants using both machine-coded algorithms and their own creative input.

6 Conclusion

In this paper we proposed some applications of computer-based tools supporting the sharing and archiving of visual information. First, we focussed on new forms of interactive maps, outlining a visual language for the preparation of historic maps and the dynamic presentation of historic developments. Further potential applications of visual communication and information systems presented include eLearning settings centered around digital pictorial lexicons, electronic support in first-aid situations and for assembly or repair, and platforms to assist people with special needs in their communication activities, including illiterates and pre-literate children. Movies will also benefit heavily from the availability of new visual technologies, as users will be able to control the levels of abstraction of dynamic scenes.

Some of the proposals presented here will be pursued in specific projects in the framework of MIRACLE.

References

[AA99] Andrienko, G.L., Andrienko, N.V.: Interactive Maps for Visual Data Exploration. International Journal of Geographical Information Science, vol. 13 (4), June 1999, pp.355-374.

[ACP98] Albacete P.L., Chang S.K., Polese G.: Iconic Language Design for People with Significant Speech and Multiple Impairments. In Mittel V.O. (ed.): Assistive Technology and AI, Springer LNAI 1458 (1998), pp. 12-32.

[BB96] Buzan, Tony; Buzan, Barry: The Mind Map Book; Plume, reprint 1996.

[Bli84] Bliss, Charles K.: The Blissymbols Picture Book. Semantography. Press: Sidney 1984. see also http://home.istar.ca/~bci.

[Bro02] Der Brockhaus multimedial (2003); see also http://www.brockhaus-multimedial.de.

[BS01] Bär, Hans Rudolf; Sieber, René: Towards High Standard Interactive Atlases - The "GIS and Multimedia Cartography" Approach. International Cartographic Conference, Beijing China, 2001; see also http://www.atlasofswitzerland.ch/pdf/ICC99_BaerSieber.pdf

[dtv91] dtv-Atlas zur Weltgeschichte, 2 volumes. Deutscher Taschenbuch Verlag; 25th edition 1991.

[DV01] DeVaul, Richard W.; Schwartz, Steven J.; Pentland, Alex "Sandy": MIThril: context-aware computing for daily life; The Media Laboratory, MIT; http://www.media.mit.edu/wearables/mithril/MIThril.pdf

[Fei02] Feiner, Steven K.: Augmented Reality A New Way of Seeing; Scientific Amercian, April 2002, issue 4/24/02; see also www.sciam.com/article.cfm?articleID=0006378C-CDE1-1CC6-B4A8809EC588EEDF

[Ge00] Gershenfeld, Neil A.: When Things Start to Think; Owl Books 2000

[Gre92] Green, Martin I.: Erste Hilfe für dein Kind; Weltbild Verlag 1992; English original: A Sigh of Relief; Bantam Books, New York 1977.

[Ho00] Hollosi, Arno: Alternative Representations and Beyond: a New Proposal for a Multi-Sensory Language Interface (MUSLI). Master's Thesis in Telematics, IICM, Graz, Austria, 2000.

[Len95] Lennon, Jennifer: Mental imagery and visualization: From the mind's machine to the computer screen; in H. Maurer, (ed.): Proc. ED-MEDIA'95, AACE, Virginia (1995), 27-32.

[ML94] Maurer, Hermann; Lennon, Jennifer: MUSLI - A MUlti-Sensory Language Interface; Proc. ED-MEDIA' 94, AACE, Virginia (1994), 341-348.

[MO03] Maurer, Hermann; Oliver, Ron: The future of PCs and implications on society; J.UCS Vol 9, No 4 (2003), to appear

[MSC03] Maurer, Hermann; Stubenrauch, Robert; Camhy, Daniela: Foundations of MIRACLE - Multimedia Information Repository, A Computer-supported Language Effort; J.UCS Vol 9, No 4 (2003), to appear

[OD95] The Oxford-Duden Pictorial English Dictionary; Oxford University Press and Duden Verlag, 1995; a series of other variants based on various languages (uni- and bilingual) is available.

[Ovi96] Oviatt, Sharon: Multimodal Interfaces for Dynamic Interactive Maps. Proceedings of CHI 96, Vancouver, Canada 1996.

[Pet95] Peterson, M.P: Interactive and Animated Cartography, Prentice-Hall, 1995, 464 pp.