A Cross-Disciplinary Bibliography on Visual Languages for Information Sharing and Archiving

Abstract: This bibliography offers citations for people who are interested in learning more about visual language, new types of communicating and archiving information with emphases on novel technologies and theoretical works in these multidisciplinary areas. This bibliography is considered in its broadest sense and covers references of research in humanities and social sciences as well as computer technology. Far from being exhaustive, it nevertheless covers essential resources in a selective way, so that the material can provide starting points for many different directions. What is not included here are references to visual programming languages.

Keywords: visual languages, visual communication, constructed languages, sign languages, interactive maps, computer-supported communication, information archiving, information retrieval, language independent communication

Categories: A.2, H.3.7, H.4.3, H.5.1, J.4, J.5

A Introduction

In [MSC03] we gave an introduction in the scientific backgrounds and the historic development and significance of various aspects of visual communication. We also investigated current and future computer technologies with respect to their potential to support visual person-to-person communication and archiving of visual information. In the course of this work we found that very little cross-disciplinary research has been done so far on these issues and also, that in our own work we had just "scratched the surface". However, vast resources exist in numerous fields that are worth to be considered when acting in this area.

The purpose of this bibliography is to make available a first list of what we think is the most important literature on the relevant topics. We cover wide ranges in a number of "thematic dimensions", from foundations to history to applications, to technical development and its implications. In general, we attached great importance to a well-balanced presentation of resources from the main fields of humanities and technology.

Because the materials listed here can very often be assigned to more than one field, categorization (not to mention restriction to the essential) was not an easy task. In our approximation we followed pragmatic considerations and always kept in mind the focus and context of this work: the purpose of this bibliography is to support the investigation of the computer as a means for new (visual) way of working with and sharing of information.

Naturally we focussed on English literature but - with German being the mother tongue of both authors - we nevertheless included a number German references as well as a few French ones.

B Structure and Categories of the Bibliography

1 Humanities and Social Sciences

Visual information and communication plays an important part in our lives. In the last years we have been more and more introduced to a multi-media world. We have been exposed to TV, video and computer, to pictures, maps, charts, matrices and diagrams and many other visual models. Research in humanities and social sciences show that new communication technologies will have far-reaching consequences. So the issue of visual language is not just one of technology.

The growing number of publications in this fields show an increasing interest in the theoretical investigation of visual languages as well as the applicability of these theories. Research on visual languages is widespread among different disciplines like philosophy, linguistics, psychology, semiotics, neurophysiology, and cognitive science.

1.1 Philosophy and Psychology

This section covers philosophical issues as well as basic aspects related to the perception of visual information and how it is organized by the human mind. Naturally, there are numerous cross-relations between these aspects.

1.1.1 Philosophy of Language

This is a very brief selection of philosophical works, dealing with philosophy of language, a field that has expanded extraordinarily in the last century. Not only foundational and conceptual questions arise but philosophical problems concerned with meaning, communication, truth, representation, the connections between mind, language and issues about philosophical methodology - it is a rich and fascinating field. This section includes mainly classical works on philosophy of language. Under the assumption that any visual communication has some relation to language, these fields are relevant in our specific context.

Austin, J.L.: How to do Things with Words. 2nd Edition. Oxford University Press, Oxford 1976.

Carnap, Rudolf : Logische Syntax der Sprache. Julius Springer-Verlag; English translation: The Logical Syntax of Language, Humanities Press, New York 1937.

Carnap, Rudolf: Meaning and Necessity. University of Chicago Press, Chicago 1947. Second edition 1956.

Dennent, Daniel C.: The Role of Language in Intelligence. In: Khalfa, Jean (ed.): What is Intelligence? The Darwin College Lectures. Cambridge Univ. Press, Cambridge 1994.

Fodor, Jerry Alan; Katz, Jerrold J. (eds.): The Structure of Language - Readings in the Philosophy of Language. Prentice-Hall, Englewood Cliffs, New York 1964. p. 612.

Katz, Jerrold J.: The Philosophy of Language. Harper & Row, New York 1966.

Ryle, Gilbert: The Concept of Mind. The University of Chicago Press, Chicago 1949.

Searle, John R.: Speech Acts. An Essay in the Philosophy of Language. Cambridge University Press, Cambridge 1969.

Wittgenstein, Ludwig: Tractatus logico-philosophicus. Routledge & Kegan Paul, London 1960.

Wittgenstein, Ludwig: Philosophical Investigations. Basil Blackwell, Oxford 1953.

Wittgenstein, Ludwig: Philosophical Grammar. (ed. by Rush Rhees).Translated by Anthony Kenny. Blackwell, Oxford 1974.

1.1.2 Mind, Cognition and Perception, With Focus on Visual Aspects

This section covers literature on the subject of mind, cognition, perception, mental representation, image and brain. The exponentially growing interest in these fields includes research in many disciplines and is opening new fascinating horizons. Books and articles therefore span disciplines like philosophy, psychology, neuroscience, linguistics and cognitive science.

Allott, Robin: The Motor Theory of Language Origin. Book Guild, 1989.

Allott, Robin: The Natural Origin of Language: The Structural Inter-relation of Language, Visual Perception and Action. Able Publishing, 2001.

Biederman, Irving: Recognition-by-components: A Theory of Human Image Understanding. Psychological Review 94, 1987. pp. 115-147.

Cornoldi, Cesrea; McDaniel, Mark A. (eds): Imagery and Cognition. Springer-Verlag, New York 1991.

Damasio, Antonio: Descartes' Error - Emotion, Reason, and the Human Brain. Grosset/Putnam, New York 1994.

Deacon, Terrence W.: The Symbolic Species: The Co-Evolution of Language and the Brain. W.W. Norton & Company, New York 1997.

Denis, Michael: Imagery and Thinking. In: Cornoldi, N.; McDaniel (eds.): Imagery and Cognition, 1991. pp.103-132.

Dondis, Donis A.: A Primer of Visual Literacy. MIT Press, Cambridge, MA 1973.

Dopkins, Stephen: The Role of Imagery in the Mental Representation of Negative Sentences. In: American Journal of Psychology 109.4, 1996. pp. 551-565.

Evans, Michael: The Geometry of the Mind. Architectural Association Quarterly 12 (4), 1980. pp. 32-55.

Haber, Ralph Norman; Hershenson, Maurice: The Psychology of Visual Perception. Holt, Rinehart and Winston, New York 1973.

Hurford, Jim R.: Language and Number - The Emergence of a Cognitive System. Basil Blackwell, New York 1987.

Jakobson, Roman: About the Relation between Visual and Auditory Signs. In: Wathedunn, W. (ed.): Models for the Perception of Speech and Visual Form. The MIT Press, Cambridge, Mass 1967.

Johnson, Mark: The Body in the Mind. University of Chicago Press, Chicago 1987.

Kennedy, John M.: A Psychology of Picture Perception: Images and Information. Jossey-Bass, San Francisco 1974. The Monist, Apr. 1974 issue ("Languages of Art").

Kosslyn, Stephen Michael: Image and Mind. Harvard University Press, Cambridge, MA 1980.

Kosslyn, Stephen Michael: Ghosts in the Mind's Machine. W. W. Norton, New York 1983.

Kosslyn, Stephen Michael: Image and Brain - The Resolution of the Imagery Debate. MIT Press, Cambridge, MA 1994.

Larkin, Jill H.; Simon, Herbert A.: Why a Diagram is (Sometimes) Worth Ten Thousand Words. Cognitive Science 11, 1987. pp. 65-69.

Mandl, Heinz: Levin, Joel (eds.): Knowledge Acquisition from Text and Pictures. North-Holland, Amsterdam 1989.

Mayer, Richard E.; Gallini, Joan K.: When is an Illustration Worth Ten Thousand Words? In: Journal of Educational Psychology. 82 (4) 1990. pp. 715-726.

Paivio, Allan: Imagery and Verbal Processes. Holt, Rinehart and Winston, New York 1971. (Reprinted 1979, Hillsdale, New York: Lawrence Erlbaum Associates).

Paivio, Allan: Images in Mind: The Evolution of a Theory. Harvester Wheatsheaf, New York 1991.

Paivio, Allan: Imagery and Memory. In: Gazzaniga, M.S. (ed.): The Cognitive Neurosciences. MIT Press, Cambridge, MA. 1996.

Petre, Marian; Green, Thomas R.G.: Learning to Read Graphics: Some Evidence that 'Seeing' an Information Display is an Acquired Skill. In: Journal of Visual Languages and Computing. (4) 1993. pp. 55-70.

Petre, Marian: Why Looking Isn't Always Seeing: Readership Skills and Graphical Programming. Communications of the ACM, 38.6, 1995. pp. 33-44.

Pinker, Steven: The Language Instinct. The New Science of Language and Mind. Penguin, Harmondsworth 1994.

Pinker, Steven: The Blank Slate: The Modern Denial of Human Nature. Viking, New York 2002.

Pinker, Steven; Ullman, Michael: The Past and Future of the Past Tense. Trends in Cognitive Science, 6, 2002. pp. 456-463.

Price, Henry. H.: Thinking and Experience. Hutchinson's Universal Library, London 1953.

Roskos-Ewoldsen, Beverly; Intons-Peterson, Margaret Jean; Anderson, Rita E. (eds.): Imagery, Creativity and Discovery: A Cognitive Perspective. Elsevier Science Pub. B.V., Amsterdam 1993.

Russell, Bertrand: On Propositions - What They Are and How They Mean. London 1919.

Smith, E.E.: Theories of Semantic Memory. In: Estes, W.K. (ed.): Handbook of Learning and Cognitive Processes . Vol. 5. Erlbaum, Hillsdale, New York 1978.

Solso, Robert L: Cognition and Vision. London, England and Cambridge, Mass. 1994.

Vygotsky, Lev S.: Thought and Language, 1934. In: Kozulin, Alex(ed.), the MIT Press, Cambridge, Mass and London 1986: Research in Humanities Computing. Vol. 1. Clarendon Press, Oxford 1986.

1.2 Visual Thinking, Visual Literacy, Media Science

Can visual tools help to make thinking visible? Can visual language change the ways we understand the world? There are some books and articles that deal with visual thinking and global communication in the 21st century. Some of them are practical guides; others include research on history and practice of visual language. In some cases these aspects are put into wider contexts.

Arnheim, Rudolf: Visual Thinking. Faber and Faber, London 1970.

Berger, Arthur Asa: Seeing Is Believing: An Introduction to Visual Communication. Mayfield, Mountain View, CA 1989, 2. kiad. 1998.

Buzan Tony, Buzan Barry: The Mind Map Book. Plume, reprint 1996.

Cavigliolo, Oliver; Harris, Ian; Tindall, Bill: Thinking Skills & Eye Q. Network Educational Press Ltd, 2002.

Hoffmann, Donald: Visual Intelligence. Norton, New York 1998.

Horn, Robert E.: Visual Language - Global Communication for the 21st Century. Marco VUP Press, 1998.

McLuhan, Marshall: Gutenberg Galaxy - The Making of Typographic Man. University of Toronto Press, 1962.

McLuhan, Marshall: Understanding Media - The Extensions of Man. McGraw-Hill 1964.

McLuhan, Marshall: Powers, Bruce R.: The Global Village - Transformations in World Life and Media in the 21st Century. University Press, Oxford 1988.

Nyíri, Kristóf: Communications in the 21st Century. Proceeding of 2002 conference; a German version is available in Nyíri, Kristóf (ed.): Allzeit zuhanden. Gemeinschaft und Erkenntnis im Mobilzeitalter. Passagen Verlag, Wien 2002.

Rheingold, Howard: The Virtual Community - Homesteading on the Electronic Frontier. MIT Press, revised edition 2000; see also http://www.rheingold.com/vc/book

Sellen, Abigail J.; Harper, Richard: The Myth of the Paperless Office. MIT Press

1.3 Semiotics

Semiotics, a philosophical theory of the functions of signs and symbols, is one of the basic disciplines in the humanities - it is connected with all kinds of communications. The theory and study of signs and symbols is essential, especially regarding elements of language but also for other systems of communication.

1.3.1 General Aspects and Basics

The following is a summary of books and articles which give a good introduction to semiotics.

Barthes, Roland: Elements of Semiology. Jonathan Cape, London 1967.

Burks, Arthur: Icon, Index, Symbol. In: Philosophy and Phenomenological Research. IX:4, 1949. pp. 673-689.

Chandler, Daniel: Semiotics for Beginners. Routledge 2002.

Eco, Umberto: Zeichen - Einführung in einen Begriff und seine Geschichte. Suhrkamp 1977. p. 203. Italian original: Il Segno. 2nd edition, Mondadori Milano 1971.

Eco, Umberto: Theory of Semiotics. Indiana University Press, Bloomington, Indiana 1976.

Eco, Umberto: Semiotics and the Philosophy of Language. University Press Bloomington, Indiana 1984.

Greimas, A.J.; Corutés, J.: Sémiotique. Dictionnaire raiaonné de la théorie du langage. Hachette, Paris 1979.

Innis, Robert E. (ed.): Semiotics: An Introductory Reader. Hutchinson, London 1986.

Johansen, Jørgen Dines: Dialogic Semiosis. Indiana University Press, Bloomington 1993.

Levin, Beth; Pinker, Steven (eds.): Lexical & Conceptual Semantics. Blackwell, Oxford 1991.

Martinus, Njhoff; Mitchell, J.T. (eds.): The Language of Images. University of Chicago Press, Chicago 1974.

Marty, Robert: Definitions of the Sign by C. S. Peirce. 1976. available at http://www.door.net/arisbe/menu/library/rsources/76defs/76defs.htm

Merrel, Floyd: Semiosis in the Postmodern Age. West Lafayette Purdue UP 1995.

Merrel, Floyd: Signs Grow: Semiosis and Life Processes. Toronto UP 1996 a.

Meunier, Jean-Guy: The Categorical Structure of Iconic Languages, Theory & Psychology. Vol. 8 (6). 1999. pp. 805-825.

Mitchell William J.T.: Iconology - Image, Text, Ideology. The University of Chicago Press, Chicago 1986.

Mitchell William J.T.: Picture Theory - Essays on Verbal and Visual Representation. The University of Chicago Press, Chicago 1995.

McKim, Robert H.: Experiences in Visual Thinking. Brooks/Cole, Monterey, CA 1972.

Moles, Abraham: L'image - communication fonctionelle. Casterman, Bruxelles 1981.

Nöth, Winfried: Semiotik. Niemeyer Verlag, Tübinger 1975.

Nöth, Winfried: Dynamik semiotischer Systeme. Metzlersche Verlagsbuchhandlung, Stuttgart 1977.

Peirce, Charles Sanders: Collected Papers. Harvard University Press, Cambridge, MA 1931. (excerpted in Buckler, Justus, ed., Philosophical Writings of Peirce, New York, Dover 1955).

Peirce, Charles Sanders: Peirce on Signs: Writings on Semiotic, edited by James Hoopes (ed.). University of North Carolina Press, Chapel Hill, NC 1991.

Saint-Martin, Fernande: Semiotics of Visual Language. Indiana University Press, Bloomington, IN 1987.

Saint-Martin, Fernande: La théorie de la Gestalt et l'art visuel. Presse de l´Université du Québec, Québec 1990.

Saussure, Ferdinand de: Course in General Linguistics. 1916.

Sebeok, Thomas A.: Semiotics as a Bridge Between Humanities and Sciences. In: Perron, Paul; Sbrocchi, Leonard G.; Colilli, Paul; Danesi, Marcel (eds.): Semiotica and Information Sciences. Legas Press, Ottawa 2000. pp.76 -100.

Sebeok, Thomas A.: Signs, Bridges, Origins. In: Global Semiotics. Bloomington Indiana Press, Bloomington 2001. pp. 9-73.

1.3.2. Pictorial Semiotics and Visual Rhetoric

Pictorial semiotics is a quite new field. It concentrates on the semiotic character of pictures and is concerned with understanding the nature and specification of such meanings which are identified with the term "picture".

Bertin, Jacques: Semiology of Graphics: Diagrams, Networks and Maps. University of Wisconsin Press, Madison 1983.

Bierman, Arthur K.: That there are no Iconic Signs. In: Philosophy and Phenomenological Research, XXIII, 2, 1963. pp. 243-249.

Boehm, Gottfried (ed.): Was ist ein Bild? Wilhelm Fink Verlag, München 1994.

Bonsiepe, Gui: Visual/Verbal Rhetoric. Dot Zero 2. Ulm 14/15/16 1966. pp. 23-40.

Carani, Marie: Sémiotique de la perspective picturale. In: Protée, 16:1-2, 1988. pp 171-181.

Carani, Marie: Sémiotique de l'abstraction picturale. In: Semiotica, 67:12, pp. 1-38.

Espe, Hartmut: Realism and some Semiotic Functions of Photographs. In: Semiotics unfolding. Proceedings of the second congress of the International Association for Semiotic Studies. Vienna 1979. Borbé, Tasso (ed.). Volume III. Mouton, Berlin, New York, & Amsterdam 1983. pp.1435-1442.

Groupe µ (Dubois, J., Edeline, Fr., Klinkenberg, J.M., Minguet, and others): Iconique et plastique: sur un fondement de la rhétorique visuelle. In: Revue d´ésthétique, 1-2, 1979. pp. 173-192.

Groupe µ, Traité du signe visuel. Pour une rhétorique de l'image. Seuil, Paris 1992.

Lindekens, René: Eléments pour une sémiotique de la photographie. Didier/Aimav, Paris & Bruxelles 1971.

Lindekens, René: Eléments de sémiotique visuelle. Klincksieck, Paris 1976.

Marcus, Aaron: Visual Rhetoric in a Pictographic-Ideographic Narrative. In: Semiotics unfolding: Proceedings of the second congress of the International Association for Semiotic Studies, Vienna, July 1979. Monton, Berlin 1983.

Schapiro, Meyer: Words, Script, and Pictures: Semiotics of Visual Language. George Braziller, New York 1996 (posthumous ed., based on essays written in the 1960s and 1970s).

Sebeok, Thomas A.: Contributions to the Doctrine of Signs. University Press, Bloomington, Indiana 1976.

Sonesson, Göran: Methods and Models in Pictorial Semiotics. The Semiotics Project. Lund 1988.

Sonesson, Göran: Semiotics of Photography. On Tracing the Index. Report 4 from the Semiotics project. Institute of Art History, Lund 1989.

Sonesson, Göran: Pictorial Concepts. Inquiries into the Semiotic Heritage and its Relevance for the Analysis of the Visual World. Lund University Press, Lund 1989.

Sonesson, Göran: The Semiotic Function and the Genesis of Pictorial Meaning. In: Center/Periphery in Representations and Institution. Proceedings of the Conference of The International Semiotics Institute, Imatra, Finland, July 16-21, 1990. E. Tarasti (ed.): Imatra: Acta Semiotica Fennica 1992. pp.156-211.

Sonesson, Göran: Pictorial Semiotics, Gestalt Theory and the Ecology of Perception. In: Semiotica 99-3/4, 1994. pp. 319-399.

Stafford, Barbara Maria: Good Looking: Essays on the Virtue of Images. MIT Press, Cambridge 1996.

1.4 Linguistics

Linguistics is a vast field. It is the scientific study of language in general and specific languages in particular. It is concerned with the study of phonetics, phonology, morphology, syntax, semantics, pragmatics, language acquisition and disorders, historical linguistics and many other important issues, like psycholinguistics and computer linguistics. A number of these sub-fields are relevant in our context of visual communication.

1.4.1 General Aspects and Basics

This section covers introductions to contemporary linguistic theories and methods of linguistic analysis. Some books draw the attention on language, cognition and mind.

Bodmer, Frederick: The Loom of Language. George Allen & Unwin, London 1944; current issue W.W. Norton & Company 1985.

Bybee, Joan; Perkins, Revere; Pagliuca, William: The Evolution of Grammar: Tense, Aspect and Modality in the Languages of the World. University of Chicago Press, Chicago 1994.

Chomsky, Noam: Three Models for the Description of Language. IRE Transactions on Information Theory, IT-2:3, 1956. pp. 113-124.

Chomsky, Noam: Language and Mind. Harcourt Brace Jovanovich Inc., New York 1972.

Chomsky, Noam: Reflections on Language. The New Press, New York 1975.

Davidson, Donald: The Logical Form of Action Sentences. 1967. reprinted In: Davidson, D. (ed.): Essays on Actions and Events. Clarendon Press, Oxford 1980. pp. 105-148.

Eco, Umberto: The Search for the Perfect Language. Blackwell, Oxford 1997.

Grice, H. Paul: Logic and Conversation. In: Cole, P.; Morgan, J. (eds.): Syntax and Semantics 3: Speech Acts. Academic Press, New York 1975. pp. 41-58.

Harris, Zelig: Mathematical Structures of Language. Wiley, New York 1968.

Jackendoff, Ray: Semantics and Cognition. The MIT Press, 1983.

Jackendoff, Ray: Languages of the Mind. MIT Press, Cambridge, MA 1992.

Kamp, Hans: A Theory of Truth and Semantic Representation. In: Groenendijk, T. M.; Janssen, V.; Stokhof, M. B. J. (eds.): Formal Methods in the Study of Language. J. A. G. Mathematical Centre Tracts, Amsterdam 1981. pp. 277-322.

Kay, Paul: Words and the Grammar of Context. CSLI Publications, Stanford, CA 1997.

Newmeyer, Frederick J. Linguistic Theory: Foundations. Cambridge University, Cambridge 1988. pp. 60-88.

Sampson, Geoffrey: Writing Systems - A Linguistic Introduction. Stanford University Press, 1987. (rev. 2nd Edition)

Sapir, Edward: The Status of Linguistics as a Science. Language 5:209. 1929.

Seipel, Wilfried (ed.): Der Turmbau zu Babel - Ursprung und Vielfalt von Sprache und Schrift. Exhibition catalog. 4 Volumes. Kunsthistorishes Museum Wien, Vienna 2003. p. 1340.

Spang-Hanssen, Henning: Recent Theories on the Nature of the Language Sign. Nordisk Sprogoch Kulturforlag, Copenhagen 1954.

van Wijngaarden, Aad.: Orthogonal Design and Description of a Formal language. Technical report, MR 76. Mathematisch Centrum, Amsterdam 1965.

Whorf, Benjamin Lee: Science and Linguistics. 1940. In: Language, Thought & Reality. MIT Press, Cambridge, MA 1956.

Wierzbicka, Anna: Semantics, Culture, and Cognition. Oxford University Press, Oxford 1992.

1.4.2 Sign Languages of the Deaf

Sign languages are the natural language of deaf people. This form of non-verbal communication has been developed by deaf people throughout the world. It started with simple hand gestures to express words, to the many complex sign languages throughout the world today. Latin Bibles from the 10th century already show drawings of finger spellings. As purely visual, dynamic languages that do not require any specific artificial medium they represent an intersection of linguistic and pictorial issues (including movement) that is particularly relevant in our context.

Adam, Heidemarie: Mit Gebärden und Bildsystemen kommunizieren. Ed. Bentheim, Würzburg 1996.

Armstrong, David F.: Original Signs - Gesture, Sign, and the Sources of Language; Gallaudet University Press, Washington DC 1999.

Armstrong, David F.; Stokoe, William C.; Wilcox, Sherman E.: Gesture and the Nature of Language. Cambridge University Press, Cambridge 1995.

Baker-Shenk, Charlotte; Cokely, Dennis R.: American Sign Language - A Teacher's Resource Text on Grammar and Culture. T.J. Publishers, Silver Spring 1980.

Boyes Braem, Penny: Einführung in die Gebärdensprache und ihre Erforschung. Signum, Hamburg 1995.

Brennan, Mary: Marking Time in British Sign Language. In: Kyle, J.; Woll, B. (eds.): Language in Sign: An International Perspective on Sign Language. Croom Helm, London 1983.

Fant, Lou: The American Sign Language Phrase Book. Contemporary Books 1944. p. 362.

Jeanes, Raymond C.: Dictionary of Australasian Signs for Communication with the Deaf. Victorian School for Deaf Children, Melbourne 1982.

Mallery, Garrick: Sign Language among North American Indians. 1881; reprinted: The Hague & Paris: Mouton 1972.

Mel'cuk Igor A.: Towards a Linguistic "Meaning<>Text" Model. In: F. Kiefer (ed.): Trends in Soviet Theoretical Linguistics. Reidel, Dordrecht 1973. pp.33-57.

Prillwitz, Siegmund; Leven, Regina; Zienert, Heiko; Hanke, Thomas; Henning, Jan: HamNoSys. Version 2.0; Hamburg Notation System for Sign Languages. An introductory guide. Signum Verlag, Hamburg 1989. p.46. see also HamNoSys 4.0: http://www.sign-lang.uni-hamburg.de/Projekte/HamNoSys/HNS4.0/englisch/HNS4.pdf

Skant, Andrea; Dotter, Franz et al.: Grammatik der Österreichischen Gebärdensprache. Forschungszentrum für Gebärdensprache und Gehörgeschädigtenkommunikation, Klagenfurt 2002.

Sternberg, Martin L.A.: American Sign Language - Unabridged Edition. HarperCollins, 1998. p. 983.

Stokoe, William C.: Language in Hand: Why Sign Came Before Speech. Gallaudet University Press, Washington DC 2001.

Sutton, Valerie: Lessons in Sign Writing 2002. publ. by the DAC (The Deaf Action Committee for Sign Writing); see also http://signwriting.org/lessons/lessonsw2002

Tomkins, W.: Indian Sign Language. Dover Publishers, 1969.

Umiker-Sebeok, Jean; Sebeok, Thomas A.: Monastic Sign Languages. 2 Volumes. Mouton de Gruyter, Berlin 1987.

1.4.3 Interlinguistics: Basics, Introductions and Overviews

Interlinguistics is the study of international linguistic communication from all its aspects including planned languages as international means of communication. Planned languages also known as "international artificial languages", "auxiliary languages" or "universal languages" are language systems created for the purpose of making international communication easier. This section lists materials that give a good overview and an interesting introduction to this field and we hope that this may give some inspiration regarding the specification of computer-supported visual languages.

Bausani, Alessandro: Geheim- und Universalsprachen. Kohlhammer, Stuttgart 1970. p.175.

Becker, Ulrich: Interlinguistik im Internet. In: Beihefte zu Interlinguistischen Informationen, Beiheft 2, Terminologiewissenschaftliche Aspekte der Interlinguistik. 1997. p. 44-46.

Blanke, Detlev: Internationale Plansprachen. Eine Einführung. Akademie-Verlag, Berlin 1985. p.408.

Couturat, Louis; Leau, Léopold: Histoire de la langue universelle. Les nouvelles langues internationales. Olms, Hildesheim-New York 1979. p.576. (reprint of the edition 1903 and 1907)

Crystal, David: Artificial Languages. In: The Cambridge Encyclopedia on Language. Cambridge University Press, Cambridge 1987. pp. 352-356.

Fettes, Mark: Interlinguistics on the Internet. In Gebethner et Wolff: Language Problems & Language Planning, 21, Nr. 2, 1997. pp. 170-176.

Gilbert, William: Problems of Languages Planned for International Use: Esperanto and Naturalistic Projects [Indiana]. Charters, 1971. 48 p.

Guérard, Albert Léon: A Short History of the International Language Movement. Fischer Unwin, London 1922.

Hauptenthal, Reinhard (ed.): Plansprachen. Beiträge zur Interlinguistik. Wiss. Buchgesellschaft, Darmstadt 1976.

Helfman, Elizabeth S.: Signs and Symbols Around the World. Lee & Shepard, Lothrop 1967.

Kamman, W. F.: The Problem of a Universal Language. Modern Language Journal, 26. March 1942. pp. 177-182.

Kittredge, Richard; Lehrberger, John (eds.): Sublanguage: Studies of Language in Restricted Semantic Domains. de Gruyter, New York 1982.

Large, Andrew: The Artificial Language Movement. Basil Blackwell, Oxford 1985. p. 239.

Libert, Alan: A Priori Artifical Languages (Languages of the World 24). Lincom Europa, München 2000. p. 139.

Lodowyck, Francis: The Ground-work of a New Perfect Language. London 1652; reprinted: The Scolar Press, Menston, England 1968.

Macaulay, Thomas C.: Interlanguage. Clarendon Press, Oxford 1930. p. 453.

Mattusch, Max Hans-Jürgen: Vielsprachigkeit: Fluch oder Segen für die Menschheit? Zu Fragen einer europäischen und globalen Fremdsprachenpolitik. Peter Lang, Frankfurt/M. 1999.

Pankhurst, E.; Delphos, Sylvia: The Future of International Language. Trubner, London 1927. p. 95

Pei, Mario: One Language for the World: Biblio an Tannen. New York 1968.

Pool, Jonathan: The World Language Problem. Rationality and Society 3. 1991. pp. 78-105.

Ronai, Paulo: Der Kampf gegen Babel oder Das Abenteuer der Universalsprachen. Ehrenwirth, München 1969. p. 176.

Sack, Friedrich L.: The Problem of an International Language. Washington & Edinburgh 1951. p. 35.

Sakaguchi, Alicja: Interlinguistik. Gegenstand, Ziele, Aufgaben, Methoden. Peter Lang, Frankfurt/M. 1998. p. 492.

Sapir, Edward: The Function of an International Auxiliary Language. In: Romanic Review, 1959.

Schubert, Klaus (ed.): Interlinguistics. Aspects of the Science of Planned Languages. In Trends in Linguistics. Studies and Monographs 42. Mouton de Gruyter, Berlin, New York 1989.

Schubert, Klaus (ed.): Planned Languages: From Concept to Reality. (Interface, Journal of Applied Linguistics, 15.1. (2000), 15.2. (2001). p. 257.

Shenton, Herbert N.; Sapir, Edward; Jespersen, Otto (eds.): International Communication: A Symposium on the Language Problem. Kegan Paul, Trench, Trubner, London 1931.

Slaughter, Mary M.: Universal Languages and Scientific Taxonomy in the Seventeenth Century. Cambridge University Press, Cambridge 1982.

Strasser, Gerhard F.: Lingua Universalis. Kryptologie und Theorie der Universalsprachen im 16. und 17. Jahrhundert. Harrassowitz, Wiesbaden 1988. p. 291.

Subbiondo, Joseph L.: Universal Language Schemes in Seventeenth-Century Britain. In: Encyclopedia of Language and Linguistics. Vol. 9. Pergamon, Oxford 1994. pp. 4841-4845.

Tonkin, Humphrey (ed.): Esperanto, Interlinguistics, & Planned Languages. University Press of America, Lanham, New York, London 1997.

1.4.4 Interlinguistics: Specific Artificial Written/Spoken Languages

During the past centuries hundreds of planned languages have been published. This section only provides an overview of some of the best known cases that might be relevant in our context. Defining languages is also an amateur activity of quite a large community which is documented by hundreds related resources on the Internet.

Barnes, Myra Edwards: Linguistics and Languages in Science Fiction - Fantasy. Arno Press, New York 1974. p. 196.

Cowan, John Woldemar: The Complete Lojban Language. The Logical Language Group, 2003. p. 620.

Forster, Peter G.: The Esperanto Movement. (Contributions to the Sociology of Languages. 32). Mouton, The Hague- Paris-New York 1982.

Freudenthal, Hans: Lincos: Design of a Language for Cosmic Intercourse. North-Holland, Amsterdam, 1960. p. 413.

Gledhill, Christopher: The Grammar of Esperanto. A Corpus-based Description. 2. Auflage. Lincom Europa, München 2000. p. 150 .

Glosa Homepage: http://www.glosa.org

Gode, Alexander; Blair, Hugh E.: Interlingua: A Grammar of the International Language. Storm Publishers, New York 1955.

Janton, Pierre: Esperanto, Language, Literature, and Community. State University of New York Press, Albany 1993. p. 169.

Meyers, Walter E.: Aliens and Linguists. Language Study and Science Fiction. University of Georgia Press, Athens, Georgia 1980.

Moskowitz, Denis: Rikchik homepage: http://suberic.net/~dmm/rikchik/intro.html. (Rikchik is a hypothetical alien visual language based on gestures)

Nicholas, Nick; Cowan, John W.: What Is Lojban? The Logical Language Group, 2003. available at http://www.lojban.org/publications/level0.html

Nuessel, Frank: The Esperanto Language. Legas, New York - Ottawa - Toronto 2000. p. 153.

Ogden, Charles Kay: Basic English - A General Introduction with Rules and Grammar. Treber, London 1930. see also http://www.basiceng.com/

Richards, I. A.: Basic English and Its Applications. Royal Society of Arts, Journal 87. London, June 2. 1939. pp. 735-755.

Walsh, Chad: The Verb System in Basic English. American Speech, December 8. 1933. pp. 137-143.

West, Michael P.: Swenson, E. et al.: A Critical Examination of Basic English. Toronto 1934.

Zamenhof, Ludwig Lazarus.: Internationale Sprache. Vorrede und Vollständiges Lehrbuch. Gebethner et Wolff, Warschau: 1887. p. 48.

1.4.5 Simplified Languages: Baby Talk, Linguae Francae, Pidgin, Creole

This section concentrates on languages that emerge from special situations where the communicational need demands a simplification of the language used. The moment when Pidgin becomes the mother tongue of a speech community it develops more elaborated grammatical features to cover the whole spectrum of human communication. Such languages are called Creole.

Ferguson, Charles A.: Absence of Copula and the Notion of Simplicity: A study of Normal Speech, Baby Talk, Foreigner Talk, and Pidgins. In: Language Structure and Language Use. Stanford University Press, 1971. pp. 277-292.

Ferguson, Charles A.: Baby Talk in Six Languages. In: Language Structure and Language Use. Stanford University Press, 1971. pp. 113-133.

Ferguson, Charles A.: Simplified Registers and Linguistic Theory. In: Obler, L.; Wise, L. (eds.): Exceptional Language and Linguistics. Academic Press, New York 1982.

Foley, William A.: Language Birth: The Processes of Pidginization and Creolization. Chapter 9 in Linguistics. The Cambridge Survey. Volume IV. Language: Context, Newmeyer, Frederick J. (ed.): The Sociocultural. Cambridge University Press, Cambridge 1988.

Hall, Robert: Pidgin and Creole Languages. Cornell University Press, 1966.

Johnson, J. E. "Excerpts" from Basic English. The Reference Shelf. Vol. 17, No. 1. The H. W. Wilson Company, New York 1944. pp. 58-68.

Todd, Loreto: Pidgins and Creoles. Language and Society Series. Routledge & Kegan Paul Ltd, 1974.

1.5 Fine Arts and Design

Visual thinking can be encouraged through art, design and creative learning techniques. These works are introductions to media of design and fine arts, ways are shown to design concepts and languages of art.

Gombrich, Ernst H.: Art and Illusion: A Study in the Psychology of Pictorial Representation. Phaidon Press, London 1960.

Goodman, Nelson: Languages of Art: An Approach to a Theory of Symbols. Bobbs-Merril, Indianapolis 1968.

Kepes, György: Language of Vision. Paul Theobald, Chicago 1944.

Kepes, György (ed.): Education of Vision. Braziller, New York 1965. including, among others, fundamental essays by Gerald Holton ("Conveying Science by Visual Presentation") and Rudolf Arnheim.

Kepes György (ed.): Sign, Image & Symbol. Studio Vista, London 1966.

Krauss, Rosalind: The Motivation of the Sign. In: Zelevansky, Lynn (ed.): Picasso and Braque. A Symposium. The Museum of Modern Art, New York 1992. pp. 262-288.

Kress, Gunther R.; Leeuwen, Theo van: Reading Images: The Grammar of Visual Design. Routledge, London 1996.

Novitz, David: Picturing. Journal of Aesthetics and Art Criticism 34, 2. 1975.

Preziosi, Donald: Advantages and Limitations of Visual Communication. In: Krampen, M.(ed.): Visuelle Kommunikation und/oder verbale Kommunikation. Hildesheim/Hochschule der Künste, Olms Verlag, Berlin 1983. pp. 1-34.

Tonfoni, Graziella; Richardson, James: Writing as a Visual Art. Scarecrow Press, 2000.

Uspenskij, Boris: Semiotics of the Russian Icon. Peter de Ridder Press, Lisse 1976.

2 Visual Languages of Pre-Electronic Media

grown out of hieroglyphs, religious iconography and visual representations of political power, to book illustrations, scientific and business process diagrams, cartoons and animation, to modern computer-generated graphics. They have grown and spread organically and globally in ways that artificially created international spoken and written languages (like Esperanto, which was invented by a single person) have never done. Will visual languages create new possibilities for human communication and human creativity?

2.1 Basics and Overviews

This section should help to give information on history and evolution of signs and symbols as well as an overview on universally used graphics, illustrations and design.

Bowman, William J.: Graphic Communication. Wiley, New York 1968.

Craig, Malcolm: Thinking Visually. New York 2000.

Dreyfuss, Henry: Symbol Sourcebook. An Authoritative Guide to International Graphic Symbols. McGraw-Hill, New York 1972; reprint: Van Nostrand Reinhold, New York 1984.

Frutiger, Adrian: Signs and Symbols: Their Design and Meaning. Van Nostrand Reinholt, New York 1989.

Goldsmith, Evelyn: Comprehensibility of Illustration: An Analytical Model. Information Design Journal 1. Goldsmith, E., 1979.

Goldsmith, Evelyn: Research into Illustration: An Approach and a Review. Cambridge University Press, Cambridge 1984.

Holmes, Nigel; DeNeve, Rose: Designing Pictorial Symbols. Watson-Guptill, New York 1985.

Huggins, William H.; Entwisle, Doris R.: Iconic Communication: An annotated bibliography. John Hopkins University Press, 1974.

Modley, Rudolf; Lowenstein, Dyno: Pictographs and Graphs: How to Make and Use Them. Harper, New York 1952.

Orcutt, David: Homepage of WorldSign / SymbolVision: http://stats.slocanlake.com/worldsign/

Sassoon, Rosemary; Gaur, Albertine: Signs, Symbols and Icons: Pre-history to the Computer Age. Intellect Books, Exeter 1997.

Twyman, Michael: Schema for the Study of Graphic Language. In: Kolers, P. A; Wrolstad, M. E.; Bouma, H. (eds.): Processing of Visible Language. Plenum, New York 1979.

Twyman, Michael: Articulating Graphic Language: A Historical Perspective. In: Wrolstad, M. E; Fisher, D. F. (eds.): Toward a New Understanding of Literacy. Praeger, New York 1986.

2.1.1 References and Picture Dictionaries

In this sections, collections of icons and symbols as well as general-purpose picture-based dictionaries are included.

Corbeil, Jean-Claude; Archambault, Ariane: Visual Dictionary: English - French - German - Spanish. Oxford University Press, 1996.

Liungman, Carl G.: Dictionary of Symbols. (1974, in Swedish). Norton & Co., New York 1991.

Oxford-Duden Pictorial English Dictionary. Oxford University Press and Duden Verlag, 1995. A series of other variants based on various languages (uni- and bilingual) is available.

Thompson, Philip; Davenport, Peter: The Dictionary of Graphic Images. St. Martins Press, New York 1980.

Tresidder, Jack: Dictionary of Symbols: An Illustrated Guide to Traditional Images, Icons, and Emblems. Chronicle Books, San Francisco 1998. - Robert Dorling Kindersley, "Visual Encyclopedia," 1995.

2.2 Maps, Information Graphics and Graphic Recording

Maps and information graphics can be powerful tools for visualization that store, organize and communicate concepts. There are many types of visual explanations and practical applications to be represented in form of maps or graphics, like historic maps and cartography of Geographic Information System, but also statistical graphics, charts and diagrams.

Harris, Robert L.: Information Graphics: A Comprehensive Illustrated Reference. Management Graphics. Atlanta 1996.

Hoff, H. E; Geddes, L. A.: The Beginnings of Graphic Recording. Isis 53 (pt. 2, no. 173), 1962. pp. 287-324.

Holmes, Nigel: Pictorial Maps. Watson-Guptill, New York 1991.

Holmes, Nigel: The Best in Diagrammatic Graphics. Rotovision, Mies, Switzerland 1993.

Tufte, Edward: The Visual Display of Quantitative Information. Graphics Press, Chershire, Connecticut 1983.

Tufte, Edward: Visual Explanations: Images and Quantities, Evidence, and Narrative. Graphics Press, Chesire, CT 1997.

2.3 Cartoons, Comics and Animation

Picture stories, cartoons, comics and animations have established an undeniable position in the popular culture. Often comics and cartoons employ series of repetitive images and symbols and communicate through a "language" that relies on an easy understanding of the meaning and emotional impact of the image.

Eisner, Will: Comics and Sequential Art. Poorhouse Press, 1985.

McCloud, Scott: Understanding Comics. Harper, 1993.

McCloud, Scott: Reinventing Comics. Harper, 2000.

Wells, Paul: Understanding Animation. Routledge, New York, London, 1998.

2.4 Specific Pre-Electronic Visual Communication Systems

One main idea in visual communication is to replace words by graphic symbols and to invent a universal symbolism. It is very useful to look at some attempts where visual languages were created for use in different contexts, like the practical system of iconic communication called Isotypes and Blisssymbolics, nowadays used as a communication aid for persons whose speech function is severely impaired. These are probably the most widely known of all temporary visual communication systems.

Bliss, Charles K.: International Semantography. Semantography Press, Sidney 1949. reprinted in Semantography (Blissymbolics. Semantography Press, Sidney 1965.

Bliss, Charles K.: Semantography-Blissymbolics. 3rd Edition. N.S.W., Semantography-Blissymbolics Publications, Sydney 1978.

Bliss, Charles K.: The Blissymbols Picture Book. Semantography Press, Sidney 1984. see also http://home.istar.ca/~bci

Bohne, Steve; Bunar, Matt; Gardella, Steve; Hynes, Pam: Cross-Cultural Icon Test. An Interactive Qualifying Project Report, WPI. May 1996.

Edmondson, William: Abstraction and Organisation in Signs and Sign Systems. Intelligent Tutoring Media, 5(2), 1994. pp. 63-72.

Gangkofer, Manfred: Warum verstehen wir Piktogramme so leicht? Zur Psychologie und Semiotik der Piktogramme. In: Schröder, Edda; Becker, Heinz; Gangkofer, Manfred (eds.): Kommunizieren mit Bliss - Sprechen über Bliss. Paritätisches Bildungswerk Bremen, Bremen 1991. pp. 57-91.

Hartmann, Frank; Bauer, Erwin K.: Bildersprache - Otto Neurath - Visualisierungen. Wiener Universitätsverlag, 2002. p. 168.

Helfman, Elizabeth S.: Blissymbolics - Speaking Without Speech. Elsevier/Nelson books, New York 1981.

Muter, Paul: Blissymbolics, Cognition and the Handicapped. In: Hjelmquist, E.; Nilsson, L.G. (eds.): Communication and Handicap. Amsterdam 1986. pp. 233-252.

Neurath, Otto: International Picture Language. London 1936; reprinted: University of Reading: Department of Typography & Graphic Communication, 1980.

Neurath, Otto: Isotype. Instructural Science 3. 1974. pp. 127-150.

Neurath, Otto: Gesammelte bildpädagogische Schriften. Haller, Rudolf; Kinross, Robin (eds.): Verlag Hölder-Pichler-Tempsky, Wien 1991.

Twyman, Michael: The Significance of Isotype. In: Graphic Communication through Isotype. 1976.

3 Computer Technology

This section covers the core of this bibliography: all computer related issues necessary to understand, develop and apply computer technology with the aim of supporting with visual tools the communication between humans and the creation and archiving of visual materials. This includes visual digital media (with a focus on visual computer interaction) but also touches computer related language issues.

What we did not include here (or at any other place in this bibliography) are references to visual programming languages. This is a large field in itself, often hiding all other aspects of visual languages which are numerous.

3.1 Computer-Mediated Visual Interaction and Communication

This comprehensive category includes a few general references on visual aspects of human-computer interaction (HCI) and visual information systems (how can visual information be retrieved from computer systems). The focus, however, is on particular issues of graphic, iconic - in general visual - information related to computer technology. Note the two different motivations behind visual human-computer interaction. Users might need to interact with computers in order to use particular software or they might want to use a computer as a medium to communicate with fellow humans. The latter is the case we focus on but naturally the two fields often intersect heavily.

Albacete P.L.; Chang S.K.; Polese G.: Iconic Language Design for People with Significant Speech and Multiple Impairments. In: Mittel V.O. (ed.): Assistive Technology and AI. Springer LNAI 1458, 1998. pp. 12-32.

Baecker, Ronald; Small, Ian; Mander, Richard: Bringing Icons to Life: Proceedings of SIGCHI conference on Human Factors in computing Systems, CHI´91. ACM Press, New Orleans 1991. pp. 1-6.

Baker, Bruce R.: Using Images to Generate Speech. Byte, 11 (3). 1986. pp. 160-168.

Baker, Bruce R.: Semantic Compaction. An Approach to a Formal Definition. In: Proceedings. 6th Annual European Minspeak Conference, Nottingham, June 1994. pp. 1-5.

Barker, Philip: Human Communication Processes. In: Yazdani, Masoud; Barker, Philip (eds): Iconic Communication. Intellect Books, 2000. pp. 1-16.

Barker, Philip; van Schaik, Paul: Designing and Evaluating Icons. In: Yazdani, M.; Barker, P. (eds): Iconic Communication. Intellect Books, 2000. pp. 161-177.

Barker, Philip; van Schaik, Paul: Icons in the Mind. In: Yazdani, Masoud; Barker, Philip (eds): Iconic Communication. Intellect Books, 2000. pp. 143-160.

Barrow H.; Baker B: Minspeak. A Semantic Compaction System that Makes Self-expression Easier for Communicatively Disabled Individuals. Byte. Vol 7, No 9. 1982. pp. 186-202.

Batini, C.; Catarci, T.; Costabile, M. F.; Levialdi, S.: Visual Strategies for Querying Databases. In: Proc. 1991 IEEE Workshop on Visual Languages. IEEE Computer Society Press, Kobe, Japan 1991. pp. 183-189.

Beardon, Colin: CD-Icon: An Iconic Language Based on Conceptual Dependency, Intelligent Tutoring Media. Vol. 3, No 4 (November 1992). 1992. pp. 111-116.

Beardon, Colin; Dormann C.; Mealing S.;Yazdani M.: Talking with Pictures. Exploring the Possibilities of Iconic Communication. Association for Learning Technology Journal. Vol 1, No. 1. 1993. pp. 26-39.

Beardon, Colin: Computer-based Iconic Communication. In: Ryan, K.; Sutcliffe, R. (eds): AI and Cognitive Science `92. Springer Verlag, London 1993. pp. 263-276. See also http://www.intellectbooks.com/iconic/iconcomm/iconcomm.htm

Beardon, Colin: Discourse structures in iconic communication. Artificial Intelligence Review, 9:2-3. 1995.

Beardon, Colin: Icon Text: An Exploration of the Limitations of Iconic Languages. In: Yazdani, Masoud; Barker, Philip (eds): Iconic Communication. Intellect Books, 2000. pp. 79-91.

Becker, Lee; Leemans Paul: VIL - A Visual Inter Lingua: Third International Conference on Iconic Communication, Bristol 1999; reprinted in: Yazdani, Masoud; Barker Philip (eds.): Iconic Communication. Intellect Books, 2000. pp.126- 142. see also http://www.es.wpi.edu/nemleem/IC99Paper.htm

Bigotta, E.; Fiorito M.; Pantano P.: A Formalized Visual Language. Bollettino del CIC, Numero 1. 1993. see also http://galileo.cincom.unical.it/Pubblicazioni/editoria/period/Numeri/quad1htm/Quad12.html

Bordogna, G.; Gagliardi, I.; Merelli, D.; Mussio, P.; Padula, M.; Protti, M.: Iconic Queries on Pictorial Data. In: Proceedings 1989 IEEE Workshop on Visual Languages. IEEE Computer Society Press, 1989. pp. 38-42.

Bretan, I.; Nilsson, R.; Harmmarstrom, K. S.: A Visual Query Language for a Multimodal Environment. In: Proceedings of CHI'94. Boston 1994.

Buxton, William: The Natural Language of Interaction. A Perspective on Non-Verbal Dialogues. In: Laurel, Brenda (ed.). The Art of Human-Computer Interface Design, Reading, MA. Addison-Wesley, 1990. pp. 405-416. available at http://www.billbuxton.com/natural.html

Catarci, Tiziana; Costabile, Maria Francesca; Levialdi, Stefano; Batini, Carlo: Visual Query Systems for Databases. A Survey. In: Journal of Visual Languages and Computing. Vol. 8, No. 2. 1997. pp. 215-260.

Champoux, Bernard; Fujisawa, Katsuki; Inoue, Tomoyoshi; Iwadate, Yuichi: Transmitting Visual Information. Icons Become Words. Proceedings of IEEE Symposium on Information Visualization 2000 (InfoVis 2000): p. 244; see also http://www.mic.atr.co.jp/organization/dept3/papers/Cails/cails_paper.html

Champoux, Bernard; David, Martin; Huot, Alain: CAILS: A Prototype for a Computer Assisted Iconic Language System. Proceedings of 2000 IEEE International Symposium on Visual Languages (VL'00). Pp. 235-234.

Chang, Shi-Kuo; Tadao, Ichikawa; Ligomenides, Panox A. (eds.): Visual Languages. Plenum Press, New York 1986.

Chang, Shi-Kuo: Visual languages and iconic languages. In: Chang, Shi-Kuo; Tadao, Ichikawa; Ligomenides, Panox A. (eds.): Visual Languages. Plenum Press, New York 1986. pp. 1-7.

Chang, Shi-Kuo: Icon Semantics - A Formal Approach to Icon System Design. International Journal of Pattern Recognition and Artificial Intelligence. Vol. 1, No. 1. 1987. pp. 103-120.

Chang, Shi-Kuo; Costagliola, Gennaro; Orefice, Sergio; ,Polese, Giuseppe; Baker, Bruce R.: A Methodology for Iconic Language Design with Application to Augmentative Communication. In: Proceedings of 1992 IEEE Workshop on Visual Languages IEEE Computer Society Press, Seattle, Washington 1992. pp. 110-116.

Chang, Shi-Kuo; Orefice, Sergio; Polese, Giuseppe; Baker, Bruce R.: Deriving the Meaning of Iconic Sentences for Augmentative Communication. In: Proceedings of the 1993 IEEE Workshop on Visual Languages, Bergen, Norway 1993. pp. 267-274.

Chang, Shi-Kuo; Orefice, Sergio; Polese, Giuseppe; Hoover, J. M.: Toward a Formal Theory of Icon Semantics with Application to Augmentative Communication. In: Proceedings of the RESNA '93 Annual Conference, Las Vegas. Resna Press: Arlington, VA 1993. pp. 130-132.

Chang, Shi-Kuo, Polese, Giuseppe; Orefice, Sergio; Tucci, Maurizio: A Methodology and Interactive Environment for Iconic Language Design. Intl. Journal of Human Computer Interaction. Vol. 14. 1994. pp. 683-716.

Chang, Shi-Kuo; Costagliola, Gennaro; Pacini, Giuliano; Tucci, Maurizio; Tortora, Genveffa; Yu, Bing; Yu, Jing-Sheng: Visual Language System for User Interfaces. IEEE Software, March 1995. pp. 33-44.

Chang, Shi-Kuo: Extending Visual Languages for Multimedia; IEEE Multimedia Fall. Vol. 3, No. 3. 1996. pp.18-26; see also http://www.cs.pitt.edu/~chang/365/vltaomm.html

Chang, Shi-Kuo: Recent Advances in Visual Information Systems. 5th International Conference. In: Proceedings of VISUAL 2002, Hsin Chu, Taiwan, March 11-13, 2002. Proceedings. Springer-Verlag, New York 2002.

Costagliola, Gennaro: On the Generation of Interactive Iconic Environments. Int. J. Human-Computer Studies 50. 1999. pp. 363-390.

Crimi, Claudia; Guercio, Angela; Pacini, Giuliano; Tortora, Genoveffa; Tucci, Maurizio: Automating Visual Language Generation. IEEE Transactions on Software Engineering. Vol. 16, No.10. October 1990. pp. 1122-1135.

Dormann, Claire: Self-Explaining Icons. In: Intelligent Tutoring Media. Vol 5, No 2. 1994. pp. 81-85; see also http://www.intellectbooks.com/iconic/self/self.htm.

Gittins, David: Icon-based Human-computer Interaction. In: International Journal for Man-Machine Studies 24, 1986. pp. 519-543.

Hollosi, Arno: Alternative Representations and Beyond: A New Proposal for a Multi-Sensory Language Interface (MUSLI). Master´s Thesis in Telematics. University of Technology, Graz 2000; see also http://www.iicm.edu/thesis/ahollosi

Honeywill, Paul: Visual Language for the World Wide Web. Intellect Books, Exeter, UK, 1999.

Honeywill, Paul: A Comparison Between Maya Hieroglyphs and Computer Icons. In: AI and Society 2000; see also http://www.w3icons.com/pdf/iconsglyphs.pdf

Honeywill, Paul: Simple Words and Visual Metaphors. Unpublished paper available at http://w3icons.com/pdf/wordsandmetaphors.pdf

Honeywill, Paul: Evaluating Appropriate Interface Metaphors. In: Yazdani, Masoud; Barker, Philip (eds): Iconic Communication. Intellect Books, 2000.pp.178-204.

Horton, Wiliam: The Icon Book: Visual Symbols for Computer Systems and Documentation. Wiley, New York 1994.

Ingen-Housz, Timothee: The Elephant's Memory - In: Search of a Pictorial Language; Learning Technology Review, Spring/Summer 1999, Apple Computer online publication: http://www.apple.com/education/LTReview/spring99/elephant

Jonassen, David H; Goldman-Segal, Ricki; Maurer, Hermann: DynamIcons as Dynamic Graphic Interfaces. Interpreting the Meaning of a Visual Representation. Intelligent Tutoring Media. Vol. 6 (3/4). 1996. pp.149-158. see also http://www.iicm.edu/iicm_papers/dynamicons.

Kaneko, S.; Ikemoto, H.; Kusui,Y.: Approach to Designing Easy-to-understand Icons. In: Proceedings 1989 IEEE Workshop on Visual Languages. IEEE Computer Society Press, 1989. pp. 246-253.

King, Andrew J.: On the Possibility and Impossibility of a Universal Iconic Communication System. In: Yazdani, Masoud; Barker, Philip (eds): Iconic Communication. Intellect Books, 2000. pp. 17-28.

Korfhage, R. R.; Korfhage, M. A.: Criteria for Iconic Languages. In: Chang, Shi-Kuo; Ichikawa, Tadao; Ligomenides, Panox A. (eds.): Visual Languages. Plenum Press, New York 1986. pp. 207-231.

Laurel, Brenda: Computers as Theatre. Addison-Wesley 1993.

Leemans, Paul: VIL - A Visual Inter Lingua. Dissertation. Worcester Polytechnic Institute, 2001. See also http://www.wpi.edu/Pubs/ETD/Available/etd-0424101-005711

Lennon, Jennifer A.: Extending Work in Visual Languages Using MUSLI: A Multi-Sensory Language Interface. Accepted by Fifth Congress of the International Association for Semiotic Studies: Semiotics Around the World Synthesis in Diversity. Berkeley, CA 1994.

Lennon, Jennifer A.; Maurer, Hermann: MUSLI: A Multi-sensory Language Interface. In: Proceedings of ED-MEDIA 94, Vancouver, Canada, AACE, Charlottesville,VA 1994. pp. 341-348.

Lennon, Jennifer A.: Mental Imagery and Visualization: From the Mind´s Machine to the Computer Screen. In: Maurer, Hermann, (ed): Proc. ED-MEDIA 95 - World Conference on Educational Multimedia and Hypermedia., Graz, Austria, 1995. pp. 27-32

Lennon, Jennifer A.: Hypermedia Systems and Applications: World Wide Web and Beyond. Springer Verlag, Berlin, Heidelberg 1997.

Lim, I.: MUSLI editor. Computer Science Report 99. University of Auckland, NZ, 1994.

Maurer, Hermann; Lennon, Jennifer A.: MUSLI: A Hypermedia Interface for Dynamic, Interactive and Symbolic Communication. Journal of Network and Computer Applications JNCA. Vol. 24, 4. 2001. pp. 273-292.

Maurer, Hermann; Lennon, Jennifer: Augmenting Text and Voice Conversations with Dynamic, Interactive Abstractions Using P2P Networking. Journal of Network and Computer Applications JNCA. Vol. 24, 4. 2001. pp. 293-306.

Maurer, Hermann; Lennon, Jennifer A.: MUSLI: A MUlti-Sensory Language Interface. Proc.eedings of ED-MEDIA' 94, AACE, Virginia 1994. pp. 341-348.

Maurer, Hermann; Stubenrauch, Robert; Camhy, Daniela G.: Foundations of MIRACLE - Multimedia Information Repository, A Computer-supported Language Effort. Journal of Universal Computer Science, Vol 9, No 4. 2003. pp.309-348.

Mealing, Stuart; Yazdani, Masoud: Communicating Through Pictures. Department of Computer Science, University of Exeter, England 1992; see also http://www.media.uwe.ac.uk/masoud/author/icon-3/siena.htm.

Mealing, Stuart: Talking Pictures. Intelligent Tutoring Media. Vol. 2, No 2. 1991. pp. 63-69; reprinted in Yazdani, Masoud (ed.): Multilingual Multimedia - Bridging the Language Barrier with Intelligent Systems. Intellect Books, London 1993.

Mealing, Stuart; Yazdani, Masoud: A Computer-based Iconic Language. Intelligent Tutoring Media, 1(3), 1992. pp. 133-136; reprinted in Yazdani, Masoud (ed.): Multilingual Multimedia - Bridging the Language Barrier with Intelligent Systems. Intellect Books, London 1993.

Mealing, Stuart: Do you see what I'm Saying. In: Yazdani, Masoud; Barker, Philip (eds.): Iconic Communication. Intellect Books, 2000. pp.68-78.

Nadin, Mihai: Interface Design and Evaluation - Semiotic Implications. In: Hartson, R.; Hix, D. (eds.): Advances in Human-Computer Interaction. Volume 2. 1989. pp. 45-100.

Nyíri, Kristóf: Pictorial Meaning and Mobil Communication. In: Nyíri, Kristóf (ed.): Conference 2002. "Communications in the 21st Century". A German translation is available in Nyíri, Kristóf (ed.): Allzeit zuhanden. Gemeinschaft und Erkenntnis im Mobilzeitalter. Passagen Verlag, Wien 2002.

Repenning, Alex: Bending Icons. Syntactic and Semantic Transformations of Icons. In: Proceedings of Visual Languages Conference '97. St. Louis, MO, 1994. pp. 296-303. Available at http://www.cs.colorado.edu/~ralex/papers/PDF/VL94.pdf

Rogers, Yvonne: Pictorial Communication of Abstract Verbs in Relation to Human-computer Interaction. British Journal of Psychology, 78, 1987. pp. 99-112.

Rogers, Yvonne: Icons at the Interface. Interacting with Computers, 1(1), 1989. pp. 105-117.

Roscoe, John: The Limits of Iconic Communication. In: Yazdani, Masoud; Barker, Philip (eds.): Iconic Communication. Intellect Books, 2000. pp.29-41.

Sanderson, David (ed.): Smileys; O'Reilly 1993.

Stubenrauch, Robert; Camhy, Daniela G.; Lennon, Jennifer A.; Maurer, Hermann: Applications of MIRACLE - Working With Dynamic Visual Information. Journal of Universal Computer Science, Vol 9, No 4. 2003. pp. 349-367

Tanimoto, Steven L.; Runyan, M. S.: PLAY: An Iconic Programming System for Children. In: Chang, S-K.; Ichikawa, T.; Ligomenides, P. A (eds.): Visual Languages. Plenum Press, New York 1986. pp. 191-205.

Tanimoto, Steven L.; Bernardelli, Carlo E.: Extensibility in a Visual Language for Web-based Interpersonal Communication. Technical report. University of Washington, IO Art School of Via Ripetta, Seattle, Washington and Rome, Italy, 1998.

Wood, Wiliam T.; Wood Susan K.: Icons in Everyday Life. In: Visual Programming Environments. Applications and Issues. IEEE Computer Society Press, Los Alamitos, CA, 1990. pp. 257-263.

Yazdani, Masoud (ed.): Multilingual Multimedia - Bridging the Language Barrier with Intelligent Systems. Intellect Books, London 1993.

Yazdani, Masoud: Communication through Icons. In: Yazdani Masoud; Barker, Philip (eds.): Iconic Communication. Intellect Books, 2000. pp. 63-67.

Yazdani, Masoud; Barker, Philip (eds.): Iconic Communication. Intellect Books, 2000.

Zhou, Michelle X.; Feiner, Steven K.: The Representation and Use of a Visual Lexicon for Automated Graphics Generation. In: Proceedings. IJCAI '97, Nagoya, Japan, Aug 23-29 1997. pp. 1056-1062.

3.2 Sign Language Translation and Gesture Interaction

Computer technology offers translation between particular sign languages of the deaf and spoken languages (in both directions). Some of the results in this field might be relevant for the development of visual communication platforms.

Holden, Eun-Jung; Owens, Robin: Visual Sign Language Recognition. In: Klete, R.; Huang T.; Gimel´farb, G. (eds.): Multi-Image Search and Analysis. Lecture Notes in Computer Science 2032. Springer, 2001. pp. 270-287.

Kadous, Waleed: GRASP - Recognition of Australian Sign Language Using Instrumented Gloves. Master thesis. University of New South Wales, Australia, Oct 1995. see also: http://www.cse.unsw.edu.au/~waleed/thesis/thesis.html

Marsh Timothy; Watt Alan: Shape Your Imagination - Iconic Gestural-Based Interaction. IEEE Virtual Reality Annual International Symposium Atlanta, GA. IEEE Computer Society, 1998.

Schulmeister, Rolf: The ViSiCAST Project - Translation into sign language generation of sign language by virtual humans (avatars) in television, WWW and face-to-face transactions. In: Stephanidis, Constantine (ed): Universal Access in HCI. Towards an Information Society for All. (Proceedings of HCI International; Vol. 3) Erlbaum, Hillsdale, New York 2001. pp. 431-435.

Schulmeister, Rolf: ViSiCAST: Übersetzung in und Generierung von virtueller Gebärdensprache im Fernsehen und Internet. In: FIfF-Kommunikation 13: 2 2000. pp. 44-47; see also http://www.izhd.uni-hamburg.de/pdfs/ViCa.PDF

3.3 Language and Knowledge Technology

Investigation of visual communication cannot be restricted to visual phenomena. In order to understand the world and to communicate with fellows humans, the human mind relies on internal concepts that in many respects have some relation to language, even if no natural spoken language seems to be involved in the thinking process at first. Therefore language and knowledge technologies are important fields in our context. In this section we cover computer models of thought and knowledge, computer linguistics, language engineering, and related topics such as ontologies and semantic nets.

Berners-Lee, Tim; Hendler, James; Lassila, Ora: Scientific American. May 2001.

Cyre, Walling R.; Balachandar, S.; Thakar, Aniruddha: Knowledge Visualization from Conceptual Structures. In: Tepfenhart et al.: Conceptual Structures: Current Practice. Lecture Notes in AI 835. Springer-Verlag, Berlin 1994. pp. 275-292.

Davies, John (ed.) et al.: Towards the Semantic Web: Ontology-Driven Knowledge Management. John Wiley & Sons, 2003.

Fargues, Jean; Landau, Marie Claude; Dugourd, Anne; Catach, Laurent: Conceptual Graphs for Semantics and Knowledge Processing. In: IBM Journal of Research and Development 30:1, 1986. pp. 70-79.

Giunchiglia, Fausto; Serafini, Luciano: Multilanguage Hierarchical Logics or How to do without Modal Logics. In: Artificial Intelligence 65:1, 1994. pp. 29-70.

Hartley, Roger T.; Coombs, Michael J.: Reasoning with Graph Operations. In: Sowa, J. F. (ed.): Principles of Semantic Networks. Explorations in the Representation of Knowledge. Morgan Kaufmann Publishers, San Mateo, CA 1991. pp. 487-505.

Jones, S.: A Multilingual Grammar Analysing System. Technical Report. Department of Computer Science. University of Exeter, 1990.

Laycock, Donald C.; Mühlhäusler, Peter: Language Engineering: Special Languages. In: An Encyclopedia of Language. Routledge, London 1994. pp. 843-875.

Lehmann, Fritz (ed.): Semantic Networks in Artificial Intelligence. Pergamon Press, Oxford 1992. Also published as a special issue of Computers and Mathematics with Applications 23, pp. 6-9.

Ludlo N. D.: Pictorial Representations of Text: Converting Text to Pictures. PhD thesis. Dept. of Artificial Intelligence. University of Edinburgh, 1992.

Lukose, Dickson; Delugach, Harry; Keeler, Mary; Searle, Leroy; Sowa, John (eds.): Conceptual Structures. Fulfilling Peirce's Dream, Lecture Notes in AI #1257. Springer-Verlag, Berlin 1997.

Masterman, Margaret: Semantic Message Detection for Machine Translation, Using an Interlingua. Proceedings 1961 International Conf. on Machine Translation. 1961. pp. 438-475.

Matsukawa, Tomoyoshi; Yokota, Eiji: Development of the Concept Dictionary-Implementation of Lexical Knowledge. In: Pustejovsky, J.; Bergler, S. (eds.): Lexical Semantics and Knowledge Representation. Proceedings of the First SIGLEX Workshop. Chapter 21. Springer-Verlag, Berkeley, CA 1991.

McCarthy, John: Situations, Actions, and Causal Laws; reprinted in M. Minsky (ed.): Semantic Information Processing. MIT Press, Cambridge, MA 1963. pp. 410-418.

McCarthy, John: Formalizing Common Sense. Ablex, Norwood, New York 1990.

Raymond, Darrel R.: Visualizing Texts. In: Making Sense of Words. Proceedings of the Ninth Annual Conference of the UW Centre for the New OED and Text Research. UW Centre for the New OED, Waterloo, Ontario 1993. pp. 19-32.

Schank, Roger C.; Tesler, Larry G.: A Conceptual Parser for Natural Language. Proceedings IJCAI. 1969. pp. 569-578.

Schank, Roger C.: Conceptual Dependency - A Theory of Natural Language Understanding. In: Cognitive Psychology (3)4. 1972. pp. 552-631.

Schank, Roger C.: Computer Models of Thought and Language. W. H. Freeman, San Francisco 1973.

Schank, Roger C. (ed.): Conceptual Information Processing. North-Holland Publishing Co., Amsterdam 1975.

Schank, Roger C.; Lebowitz, Michael ; Birnbaum, Lawrence: An Integrated Understander. In: American Journal of Computational Linguistics 6, 1980. pp.13-30.

Schank, Roger C.; Riesbeck, Christopher K. (eds.): Inside Computer Understanding. Lawrence Erlbaum Associates, Hillsdale, New York 1981.

Schank, Roger C.: Dynamic Memory. Cambridge University Press, New York 1982.

Schank, Roger C.; Kass, Alex; Riesbeck, Christopher K.: Inside Case-Based Explanation. Lawrence Erlbaum Associates, Hillsdale, New York 1994.

Sowa, John F.: Conceptual Graphs. In: Bernus, P.; Mertins, K.; Schmidt, G. (eds.): Handbook on Architectures of Information Systems. Springer-Verlag, Berlin 1998. pp. 287-311.

Sowa, John F.: Knowledge Representation. Logical, Philosophical, and Computational Foundations. Brooks Cole Publishing Co., Pacific Grove, CA 2000.

3.4 Visual Applications of New Media

Here we touch a number of areas related to issues of new, digital media (again focussing in visual aspects), far from trying to achieve completeness. This includes digital photography and movies, and applications such as interactive maps, computer games and iconic or graphic chats involving avatars.

Andrienko, Gennady L.; Andrienko, NataliaV: Interactive Maps for Visual Data Exploration. In: International Journal of Geographical Information Science. Vol. 13 (4). June 1999. pp. 355-374.

Bär, Hans Rudolf; Sieber, René: Towards High Standard Interactive Atlases - The "GIS and Multimedia Cartography" Approach. International Cartographic Conference, Beijing China 2001. see also http://www.atlasofswitzerland.ch/pdf/ICC99_BaerSieber.pdf

Fuji, H.; Korfhage, R. R.: Features and a Model for Icon Morphological Transformation. In: Proceedings of 1991 IEEE workshop on visual languages. IEEE Computer Society Press, Kobe, Japan 1991. pp. 240-245.

Game Studies - The International Journal of Computer Game Research. Online publication at http://www.gamestudies.org

Jayasinha C.; Lennon, Jennifer A.; Maurer, Hermann: Interactive and Annotated Movies. In: Proceedings of ED-MEDIA 95, Graz, Austria 1995. pp. 366-371.

Mäyrä, Frans (ed.): Computer Games and Digital Cultures. Proceedings of the Conference CGDC 2002. Tampere University Press, Tampere, Finland 6-8 June 2002. p. 356.

Mitchell, William, J.: The Reconfigured Eye: Visual Truth in the Post-Photographic Era. MIT Press, Cambridge Mass 1994.

Oviatt, Sharon: Multimodal Interfaces for Dynamic Interactive Maps. Proceedings of CHI 96, Vancouver. Canada 1996.

Peterson, Michael P.: Interactive and Animated Cartography. Prentice-Hall 1995. p. 464.

3.5 Future of Computers: Augmented Reality and Ubiquitous Computers

This section is based on our belief that in the future, computer usage will be much more visual and intuitive than today. The materials referenced support this view.

DeVaul, Richard W.; Schwarz, Steven J.; Pentland, Alex: "Sandy": MIThril: Context-Aware Computing for Daily Life. The Media Labaratory, MIT. http://www.media.mit.edu/-wearables/mithril/MIThril.pdf

Dreyfus, Hubert L.: What Computers Still Can't Do. A Critique of Artificial Reason. MIT Press, Cambridge, MA 1992.

Feiner, Steven K.: Augmented Reality - A New Way of Seeing: Scientific American. April 2002: see also http://www.sciam.com/article/cfm?articleID=0006378C-CDEI-ICC6-B4A8809EC588EEDF

Gershenfeld, Neil A.: When Things Start to Think. Owl Books, 2000.

Maurer, Hermann; Carlson, Patricia: Computer Visualization, a Missing Organ, and a Cyber-equivalency. Collegiate Microcomputers: 10(2), 1992. pp. 110-116.

Maurer, Hermann; Oliver, Ron: The Future of PCs and Implications on Society. In: Journal of Universal Computer Science, Vol 9, No 4. 2003. pp. 300-308

Pfurtscheller, G.; Neuper, C.; Guger, C.; Harkam, W.; Ramoser, H.; Schlögl, A.; Obermaier, B.; Pregenzer, M.: Current Trends in Graz Brain-Computer Interface Resarch. IEEE Transactions Rehab. Engineering 8, 2. 2000. pp. 216-219.

References

[MSC03] Maurer, Hermann; Stubenrauch, Robert; Camhy, Daniela G.: Foundations of MIRACLE - Multimedia Information Repository, A Computer-supported Language Effort. In: Journal of Universal Computer Science, Vol 9, No 4. 2003, pp. 309-348.