Co-operative and Interactive Distance Learning: Application of Team-Oriented and Selective Learning Strategies in a European Bank

Abstract: Major companies, especially banks, invest in interactive distance learning replacing face-to-face training. Research has shown learning gains are mostly due to a shift in instruction. In this study, a WBT about currency management of a major German bank was examined. The communicational features of the WBT comprise a discussion forum, note taking, and automatic messaging of questions and answers between experts and students. The experimental design compared a face-to-face seminar with WBT learning. The results show that WBT participants learned as much as the seminar participants, but in about 70% of the seminar's study time. Young seminar participants performed better than older ones, while WBT learning did not produce an age effect. The results of the study demonstrate that the learners in the bank tend to choose traditional learning strategies, they do not cope optimally with co-operative and selective learning strategies, and they tend to appreciate audio-visual media. Experts did not voluntarily play an active role in the discussion processes. Communicational features, however, were used quite frequently. The users who were experienced in using a CBT and showed high self esteem gained most from WBT learning.

Keywords: computer mediated communication, Web based training, computer supported learning strategies

Categories: H.5.1, J.4

1 Background

Electronic performance support systems (in the form of on-line media and self-directed learning environments) are among the most effective training solutions in terms of cost, time and logistics [cf. McGraw 1994]. Therefore, some German banks have already reduced their face-to-face training courses by approximately 30% p.a. and all major banks are now introducing Web-based training (WBT) as a means for cost effective training [Hasebrook 1999a]. During the same period of time, German banks experienced a considerable increase in training costs: Major German banks and bank associations spent between 88% and 136% more money on training in 1996 than in 1989. A closer look at these data reveals a dramatic increase in training costs between 1989 and 1993. It should be noted that - in compliance with German laws - training costs are all direct and indirect costs related to training, which has been jointly released by the employees' work council and the employer. Since 1994, however, there has been a decline in training budgets at the same rate as in other German business sectors: About 75% of all employees participated in training courses, this percentage increased about 10% from 1994 to 1997. At the same time, the educational budgets were reduced by nearly 10% [Ausbilderhandbuch 1998].

Banks are spending 6% of their personnel budgets for training but 15-30% of their administrative and operational budgets for information and communication technology (ICT). For instance, the largest of German banks, Deutsche Bank, spent US-Dollar 205 million on training and US-Dollar 1.3 billion on ICT. This translates into approximately US-Dollar 2,500 for training and US-Dollar 4,000 for ICT per employee [Moormann 1999]. In 1997 Internet- and Intranet-based training accounted for only 2.4% of the total cash flow of the educational market. Johnston & Moretti estimate the annual increase in these training technologies to be 140% and 62%, respectively [Johnston & Moretti 1998]. In 2002, Internet- and Intranet-based training will represent about 40% of the educational market.

2 Co-operative Learning with Electronic Media

Meta-analyses of computer-based training show that multimedia and online media are not overly effective. Kulik and Kulik examined 248 research studies about computer-supported learning [Kulik and Kulik 1991]. 150 studies failed to show any significant effects. The other studies showed only a slight advantage of multimedia over textbooks and lectures: Error rates of simple retention tests were 5% to 15% lower than before (Eta²=.15), problem solving was hardly enhanced and study time was reduced from 100% to percentages ranging from 80% to 20%, with an average reduction of time to 70% (Eta²=.35). Considering all studies included in the meta-analysis, multimedia produced only a small effect (Eta2<.01) [Hasebrook 1995]. Clark and Craig investigated several meta-analyses [Clark and Craig 1992], including the studies of Kulik and Kulik [Kulik and Kulik 1991; Kulik, Bangert-Downs & Williams 1983; Kulik, Kulik & Cohen 1980]. They draw the following conclusions: (1) Multiple media are not the factors that influence learning. (2) The measured learning gains are most likely due to instructional methods. (3) The aspects of picture superiority and dual coding have not been supported [cf. Paivio 1986].

Additionally, multiple media are especially helpful if a well structured and fact-oriented subject matter is presented to the learners, different perspectives are offered, and self motivation and self esteem are supported [e.g. Hasebrook and Otte 2002]. Many other studies have confirmed that multimedia applications enhance learning, only if the individual skills and abilities match the demands of the learning task and the functionality of the multimedia system [e.g. Reynolds & Danserau 1990; Barba & Armstrong 1992; Mayer & Sims 1994]. Therefore, it is necessary to teach users strategies and concepts to use multimedia applications. Additionally, it is necessary to adapt the system to individual abilities and the overall learning environment [Larkin & Chabay 1992; Hasebrook & Gremm 1999].

Little is known about the effects of co-operative distance learning upon corporate culture, learning behavior, and communication processes. Several studies compared computer conferencing via e-mail, video-conferencing, telephone conferences and personal communication [Sproull & Kiesler 1991; Kiesler 1992]. These studies determined that video-conferencing is much more similar to telephoning than to personal communication.

As Sproull and Kiesler discovered, simple e-mail conferences can provide several advantages: Personal communication takes less time but electronic mailing leads to agreements more frequently [Sproull and Kiesler 1991]. Additionally, conferencing by e-mail allows for a more symmetrical participation than personal discussions. Weisband & Atwater, however, reported that self ratings of contributions were more inflated and less accurate in electronic communication than in face-to-face communication [Weisband & Atwater 1999]. The biases mainly stem from liking or disliking peers in face-to-face groups but not in electronic ones/groups.

Experts play an important role in online discussions. Ogata & Yano found out that the presence of an expert led to more direct participation in an online discussion, but also to a higher drop-out rate while peer-to-peer discussions suffered from poor active participation if the participants were not directly invited to join in the discussion by their peers [Ogata & Yano 1998]. Bolling & Robinson compared three different learning groups: (1) individual learning with printed matter, (2) co-operative team learning with printed material and special instructions, and (3) team learning without special instructions using multimedia courseware (Bolling & Robinson 1999]. Taking into account the prior knowledge of the participants, the authors found co-operative learning to be the most effective training method. Individual and multimedia team learning did not differ significantly. The best performance was observed among participants of the co-operative learning group with high prior knowledge. These and similar findings are in line with recent research results indicating that group cohesion is enhanced when group members are actively managed and master high performance barriers [Tesluk & Mathieu 1999]. Leader-Member-Exchange (LMX) produces higher follower performance as compared to transformational leadership irrespective of physical distance [Howell & Hall-Meranda 1999], and only content goals with a clear skills improvement focus have been found to support performance in training programs [Brett & VandeWalle 1999].

3 The Learning Environment

A WBT about currency management was developed by Bank Academy in charge of a major German bank. The WBT is based on the Hyperwave information server and its learning platform GENTLE [Maurer 1998]. GENTLE has now evolved into the commercial software package 'eLS' (eLearning Suite). This software stores and maintains the user interface (e.g. buttons, frames), the structure (e.g. links, hierarchy of pages) and the actual content (e.g. HTML-pages, images) separately. Thus, all complete WBT pages are composed on demand and may contain individual information, such as notes and user defined links, without interfering with the contents of the WBT delivered to other users. Special features of the learning platform were used to automatically generate specific learning paths for two different target groups and eight different experimental settings (see section 4.3).

The study reported here was conducted with this WBT. The WBT consisted of five modules comprising approximately 100 pages each. About one third of the pages contained animations or interactive exercises, such as calculators and interactive telephone orders. Important content areas, such as definitions, examples, exercises, and team instructions, were marked by special icons.

Half of the participants were automatically pooled in learning teams with five persons each by the system and the other half studied individually. All participants were allowed to take notes and write contributions to the discussion forum. All notes and contributions were typed according to their contents, that is, the user decided whether she or he wanted to type in a question, an answer, an agreement, a disagreement or a simple remark. All notes were linked to a particular phrase or page in the WBT. Additionally, different access rights could be attached to each note: Public, learning team (if available), and private. Private notes were marked with gray icons, public and team notes with green icons. All notes containing questions were sent as an e-mail to an expert who decided whether he or she wanted to respond to that question. The notes which had been responded to by an expert were marked with a blue icon. All public notes were automatically copied to the discussion forum with a link in the note enabling the user to access the anchor of the note by clicking on that link.

Figure 1: The design of the learning environment: Navigation and table of content (left), general tools (above), and an interactive audio exercise (centre) are displayed by the Hyperwave system. .

The notes did not only support the learning process by motivating the users to discuss the subject matter of the WBT. They also provided a useful source of information for the adjustment and improvement of the system, because the users took lots of notes which described technical or design problems. Furthermore, a background library of encyclopaedias and news services enabled the user to access a vast amount of background information and most recent information without leaving the WBT environment.

Automatic control of access rights, automatic generation of learning paths according to target groups and experimental conditions including all material and media involved, support of individual anchored annotations, typed links, and the integrated communication system including forum and email are the novel features of the learning platform used in this study.

4 Method

4.1 Participants

Outlets of the bank all over Germany were asked to nominate trainees of their corporate finance departments for a two-day seminar about currency management. Seventy persons were randomly assigned to the one-day WBT, thirty persons to traditional face-to-face seminars resulting in 64 complete data sets of the WBT users and 30 complete data sets of the seminar participants. Only ten of these 94 persons were female; the mean age was 35.2 years (standard deviation, SD, 11.8).

4.2 Material and Procedure

The WBT learners used the WBT described above. The WBT was completely new developed and based on the printed material, such as papers and slides, used in the seminar. Additionally, the trainer of the seminar groups served as the subject matter expert of the WBT development. Thus, the instructional methods were different but not the content bases of the training.

In the beginning, all subjects filled in a survey about personal data, that is, gender, age, professional experience, prior knowledge, WBT experience and their personal expectations. Furthermore, they responded to 16 multiple-choice questions about currency management. While learning with the WBT, the users' inputs were automatically recorded by the system. All WBT participants learned about the WBT features conducting an introductory module which took them about 20 minutes to complete. Each module started with a comprehensive instruction according to the actual experimental condition, a brief overview and offered a multiple-choice self test. After having finished a module, the WBT offered an evaluation form with questions about the correctness, job-relatedness and user-friendliness of the WBT module, which could be filled-in voluntarily.

After the training, all seminar and WBT participants filled in a second survey about their experiences with the training course and responded to a multiple-choice test with 24 questions: 16 questions were taken from the pre-test, 8 questions were newly introduced. The survey was paper and pencil work, all multiple choice questions were presented at the computer and were rated by an expert team according to their difficulty (cf. figure 2). Test and survey were filled in anonymously and without observation in order to avoid social desirability distortion [cf. Richman et al. 1999]. It took the participants about 40 minutes to fill-in the survey and respond to the multiple-choice test. The WBT course took about 8.5 hours (SD=1.1) and the seminar about 12 hours of net study time to be finished. Times were recorded automatically by the system (from log-in to log-out) or reported by the on site trainer, respectively. All WBT learners took part in a moderated team discussion about their experiences using the WBT. The results of these discussions were recorded by the moderator.

4.3 Design

The first experimental factor was the comparison of the between factor 'seminar vs. WBT learning' with respect to acceptance and performance criteria. Another set of experimental factors was realised by a mixed design within the WBT group. One half of the WBT group was automatically assigned to a learning team resulting in the between factor 'team vs. individual learning'; the teams were automatically formed according to the login time of the learners. In every second WBT module, the learners were instructed to read the overview and to take the self test prior to the access of the module and then to decide - based on the test results - whether they wanted to go through all pages or only parts of the module. This instruction resulted in the within factor 'complete vs. selective learning'. Each module contained several audio and video files and a simple text version of the same content.

The system automatically assigned the WBT users to different groups which had access to the audio-visual media in every second module. This resulted in the within factor 'text vs. av media'. All factors were counterbalanced by a Latin square procedure among the subjects. In summary, the experimental set-up of the WBT system resulted in a mixed design with the between factor 'team vs. individual learning' and the within factors 'complete vs. selective learning', and 'text vs. av media'. Additionally, qualitative data were collected by interviews with the participating experts and by team discussions after the training program.

5 Results

All survey ratings range from 1 ('very good' or 'I totally agree') to 5 ('very poor' or 'I totally disagree'). As the scores of the multiple choice items differ according to their difficulty, all test scores are expressed as percentage of the maximum score (ranging from 0% to 100%). Due to the variable cell frequencies of the design and some missing data, the General Linear Model (GLM) procedure of the SPSS statistical software package was used to analyze the data. A GLM is comparable to a normal mixed MANOVA.

5.1 Comparison of WBT and Seminar

The study time of the WBT and the seminar differed significantly (8.5 vs. 12.0 h; F[1,92]=319,9; p<.001). The statistical analyses showed main effects of the learning group in the pre-test (F[1,80]=9,3; p<0.01) and the post-test for the 16 old items (F[1,80]=5,1; p<.05), but not for the 8 new items (F[1,80]=1.7; n.s.): The WBT group started with higher test scores and showed better performance for the items from the pre-test. But there was no significant difference concerning the new items. Taking into account the pre-test scores as a co-variant, the main effect of the learning group is reduced to a weak tendency for the post-test results (F[1,80]=1.7; p<.2) and the co-variant is highly significant (F[2,90]=29.1; p<.001).

The test results showed no significant differences for female and male participants due to the small number of women, although they did slightly better than men (75.0 vs. 73.3% in the final test). The participants were grouped into four categories according to their age: 20 to 35, 36 to 45, 46 to 55, and 56 to 65 years. There is a tendency that young participants performed better in the final test than older ones (F[194,3]=2,4; p<0.1), but there was no significant difference in the pre-test results (cf. figure 2). Most importantly, there was an interaction of learning group and age group: Young seminar participants learned more than older ones, but there was no such difference within the WBT group (F[194,3]=3,2; p<.05). Table 1 and figure 3 summarize the results of the pre- and post-test as a function of gender and age.

All learners judged their prior knowledge to be on a medium level (WBT 3.5; seminar 3.6). After the training, the judgement of WBT participants concerning their knowledge was improved, but the judgement of the seminar learners was significantly better (2,7 vs. 1.3; F[1,80])=39.0; p<.001), although their test results were lower than those of the WBT learners. There were no more significant differences in the individual judgements of the WBT and the seminar group. Table 2 summarizes the scores of the individual judgements.

5.2 Factors of Online Learning

Team vs. Individual Learning. There was no significant difference of the pre- and post-test scores between team and individual learning. There are only two tendencies: Individual learning leads to slightly better acceptance of the WBT than team learning (F[1,57]=1.7; p<.2), and to a better judgement of the knowledge acquired during the training (F[1,57]=2.8; p<.1). In general, two co-variants affect the post-test results, but not the pre-test results: A high judgement of prior knowledge and experiences using a CBT lead to better post-test results (F[1,59]=13.9; p<.001 and F[1,59]=6.2; p<.05, respectively). Experienced users of the corporate Intranet, however, did not show significantly better test results (cf. figure 4).

Table 2: Individual judgements and acceptance ratings in the pre- and the post-test as a function of learning group (seminar vs. WBT); scores are ranging from 1 (very good) to 5 (very bad).

Complete vs. Selective Learning. Once again, complete and selective learning strategies did not lead to significant differences in test results and acceptance ratings. Therefore, we checked the number of page and function calls as a function of the different learning conditions. On average, 35 notes were read, seven taken and the forum was accessed 31 times per module. Each user took an average of five notes per module and additionally wrote two messages to the forum. Most of the notes were public. Selective team learners tend to use the note function more frequently than the other learners (F[1,60]=2.1; p<.2). Complete learners accessed 398 pages of the WBT and selective learners 411, group learning led to 395 page accesses and individual learning to 412 page accesses. There were no significant differences in the number of function calls and page accesses in all groups. Table 3 summarizes the data for the factors team vs. individual and complete vs. selective learning.

Effects of Audio-Visual Media. There is a tendency that learners with audio-visual media did better in the post-test with 24 items than learners without (77.3% vs. 74.7%; F[1,63]=2.7; p<.1). And there is a tendency for better acceptance of the modules with audio-visual media than those without (F[1,41]=3.2; p<.1). Table 4 summarizes the test results and acceptance data of the modules with and without audio-visual media.

There are some interesting additional results concerning the module surveys: Module 3 and 4 contained many calculations as interactive exercises while module 2 and 5 did not. Thus, module 3 and 4 got worse acceptance ratings than module 2 and 5, especially concerning their user friendliness and their job-relatedness (F[1,41]=5.2; p<.05). Furthermore, only half of the module surveys contained a direct feedback summarizing all user inputs in simple bar charts. These surveys with direct feedback collected 372 user inputs while the surveys without direct feedback collected only 312 inputs. Thus, providing direct feedback to the users of surveys seems to be an easy way to improve compliance.

Expert Participation. The experts received about ten e-mails per day during the learning phase. The questions ranged from serious questions to complaints, e.g. about the number of calculations which had to be performed by the learners. The experts wrote only 20 answers reviewing the questions they had got via e-mail and via the forum of the WBT. The average length of such an answer was about two to four sentences (40 to 80 words). In professional virtual seminars the input of experts is much higher, e.g. in a virtual seminar at the University of Maryland with 15 sessions the experts wrote about 8,000 words and the participants about 2,750 words (cf. Bernath & Rubin 1998]. However, all experts claimed to have given strong support to co-operation and team learning based on electronic discussion forum or e-mail messaging. However, there was no clear organizational procedure that enabled the experts to withdraw from their normal duties and work on the WBT, instead.

Table 3: Acceptance ratings (1 to 5) and system calls per module as a function of learning strategies within the WBT (team vs. individual learning and complete vs. selective learning).

Team Interviews and Discussion. In the team sessions after the WBT training positive and negative aspects of the WBT were collected and discussed. All participants indicated on a board whether they considered the WBT to be a very negative, negative, neutral, positive or very positive means for training. As in the surveys, the individual judgements summed up to a neutral attitude towards the WBT. Positive aspects discussed by the participants were (1) self paced and self directed learning, (2) free choice and access to information, (3) direct feedback for tests and inputs, (4) fast and efficient learning, and (5) opportunity for distant communication. Negative aspects were (1) too many and too difficult calculations, (2) too much content not directly targeted at the different departments of the company, (3) difficult handling of the calculation forms, (4) too many overviews and indices, (5) a too restricted learning time, and (6) not much input from experts.

6 Conclusions

One of the major conclusions of this study is that success does not come simply by using the latest online techniques: The learning culture of the participants and the experts involved in the WBT clearly did not support the success factors of online learning. Although the participants used navigational and communicational features quite frequently, they did not receive much input from the experts, and they did not pick up new learning strategies, such as team and selective learning. This line of reasoning is supported by strong impact of self esteem (judgement of prior knowledge) and of CBT experience on test results. The age effect indicates that WBT is offering a more equal opportunity for learning than seminars. Additionally, seminar participants considered their learning results to be better than WBT learners did, although objectively it was not.

A future study will examine a similar WBT environment. However, there will be a variable learning time which is not going to be restricted to a single day, clear instructions for the corporate departments how experts should be involved, and the introductory module will not only give a brief overview of the WBT features. The introduction will actively train communicative skills and the selection of information from comprehensive online learning environments [cf. Hasebrook 1999b].

In summary, the WBT at least reaches the performance of face-to-face seminars within a shorter period of time. Audio-visual media does not improve performance measures, but they improve acceptance ratings. WBT works fine for young and old people whereas face-to-face training does not. Although instructed to do so, WBT learners stick to traditional learning methods and have to be guided and trained to pick up new ones, such as co-operative and selective learning methods. Experienced CBT users - but not Internet users - as well as learners with high self esteem profit most from WBT learning.

Acknowledgements

I would like to thank the subjects of these/this study/ies for their active participation, the development team of the Hyperwave learning suite for their technical support, and my colleagues and five anonymous reviewers for their critical and helpful comments on earlier versions of this paper.

References

[Ausbilderhandbuch, 98] Ausbilderhandbuch: "Daten und Fakten zur bankbetrieblichen Aus- und Weiterbildung", Stuttgart (Germany): Deutscher Sparkassen Verlag, 1998.

[Barba and Armstrong, 92] Barba, R.H., Armstrong, B.E.: "The effect of HyperCard and interactive video on earth and space science students´ achievements", Journal of Educational Multimedia and Hypermedia, 1992, Vol. 1, No. 1, 323-330.

[Bernath and Rubin, 98] Bernath, U., Rubin, E.: "A virtual seminar for international professional development in distance education", Informatik Forum, 1998, Vol. 12, No.1, 18-23.

[Boling and Robinson, 99] Boling, N.C., Robinson, D.H.: "Individual study, interactive multimedia, or cooperative learning: Which activity best supplements lecture-based education?", Journal of Educational Psychology, 1999, Vol. 91, No.1, 169-174.

[Brett and VandeWalle, 99] Brett, J.F., VandeWalle, D.: "Goal orientation and goal content as predictors of performance in a training program", Journal of Applied Psychology, 1999, Vol. 84, No. 6, 863-873.

[Clark and Craig, 92] Clark, R.E., Craig, T.G.: "Research and theory on multi-media learning effects", in M. Giardina (ed.): "Interactive multimedia learning environments. Human factors and technical considerations on design issues", Heidelberg: Springer, 1992, 19-30.

[Hasebrook, 95] Hasebrook, J.: "Learning with multimedia", German Journal of Educational Psychology, 1995, Vol. 9, No. 2, 95-103.

[Hasebrook, 99a] Hasebrook, J.: "Web-based training, performance, and controlling", Journal of Network and Computer Applications, 1999a, Vol. 22, 51-64.

[Hasebrook, 99b] Hasebrook, J.P.: "Learning in electronic media: Exploring Web-based training and the human mind", Proceedings of the WebNet conference 1999, Charlottesville, VA: AACE, 1999b, 1025-1032.

[Hasebrook and Gremm, 99] Hasebrook, J., Gremm, M.: "Multimedia for vocational guidance: Effects of testing, videos, and photography on acceptance and recall", Journal of Educational Multimedia and Hypermedia, 1999, Vol. 8, No. 2, 217-240.

[Hasebrook and Otte, 02] Hasebrook, J., Otte, M.: "E-Learning im Zeitalter des E-Commerce" [e-learning the era of e-commerce, in German), Bern: Huber, 2002.

[Howell and Hall-Meranda, 99] Howell, J.M., Hall-Meranda, K.E.: "The ties that bind: The impact of Leader-Member Exchange, transformational and transactional leadership, and distance on predicted follower performance", Journal of Applied Psychology, 1999, Vol. 84, No. 5, 680-694.

[Johnston and Moretti, 98] Johnston, P., Moretti, C.: "Worldwide and U.S. IT training and education markets and trends", New York: International Data Corporation (IDC), 1998.

[Kiesler, 92] Kiesler, S.: "Talking, teaching, and learning in network groups: Lessons from research", in Kaye, A.R. ]Ed.): "Collaborative learning through computer conferencing", Heidelberg: Springer, 1992, 145-165).

[Kulik and Kulik, 91] Kulik, C.-L., Kulik, J.A.: "Effectiveness of computer-based instruction: An updated analysis", Computers in Human Behavior, 1991, Vol. 7, 75-94.

[Kulik et al., 83] Kulik, J.A., Bangert-Downs, R.L., & Williams, G.W.: "Effects of computer-based teaching on secondary school students", Journal of Educational Psychology, 1983, No. 75, 19-26.

[Kulik et al., 80] Kulik, C.-L., Kulik, J.A., Cohen, P.: "Effectiveness of computer-based college teaching: A meta-analysis of findings", Review of Educational Research, 1980, No. 50, 252-544.

[Larkin and Chabay, 92] Larkin, J.H., Chabay, R.W. (eds.): "Computer assisted instruction and intelligent tutoring systems: Shared goals and complementary approaches", Hillsdale, NJ: Erlbaum, 1992.

[Maurer, 98] Maurer, H.: "Using the WWW System Hyperwave as the Basis of a General Networked Teaching and Learning Environment", Journal of Computing and Information Technology, 1998, Vol. 6, No. 1, 63-72.

[Mayer and Sims, 94] Mayer, R.E., Sims, V.K.: "For whom is a picture worth a thousand words? Extensions of a dual-coding theory of multimedia", Journal of Educational Psychology, 1994, Vol. 86, No. 3, 389-401.

[McGraw, 94] McGraw, K.L.: "Performance support systems: Integrating AI, hypermedia, and CBT to enhance user performance", Journal of Artificial Intelligence in Education, 1994, Vol. 5, No. 1, 3-26.

[Moormann, 99] Moormann, J.: "Umbruch in der Bankinformatik - Status Quo und Perspektiven für eine Neugestaltung", in Moormann, J., T. Fischer (eds.), Handbuch Informationstechnologie in Banken, Wiesbaden: Gabler, 1999, 3-20.

[Ogata and Yano, 98] Ogata, H., Yano, Y.: "Supporting awareness for augmenting participation in collaborative learning", Proceedings of the WebNet98, Charlottesville: AACE., 1998, 1040-1045.

[Paivio, 86] Paivio, A.: "Mental representations: A dual-coding approach", New York, Oxford: Oxford University Press, 1986.

[Reynolds and Dansereau, 90] Reynolds, S.B., Dansereau, D.F.: "The knowledge hypermap: An alternative to hypertext", Computers & Education, 1990, Vol. 14, 409-416.

[Richman et al., 99] Richman, W.L., Kiesler, S., Weisband, S., Drasgow, F.: "A meta-analytical study of social desirability distortion in computer-administrated questionnaires, traditional questionnaires, and interviews", Journal of Applied Psychology, 1999, Vol. 84, No. 5, 754-775.

[Sproull and Kiesler, 91] Sproull, L., Kiesler, S.: "Connections: New ways of working in the networked organization", Cambridge, MA: MIT Press, 1991.

[Tesluk and Mathieu, 99] Tesluk, P.E., Mathieu, J.E.: "Overcoming roadblocks to effectiveness: Incorporating management of performance barriers into models of workgroup effectiveness", Journal of Applied Psychology, 1999, Vol. 84, No. 2, 200-217.

[Weisband and Atwater, 99] Weisband, S., Atwater, L.: "Evaluating self and others in electronic and face-to-face groups", Journal of Applied Psychology, 1999, Vol. 84, No. 4, 632-639.