| Submission Procedure |
|
|
|
|||||||||||||||
Teaching on the WWW: Assignment Focus and Information Indexing
School of Electrical Engineering and Computer Science Washington State University Pullman, WA 99164-2752 USA email rada@eecs.wsu.edu Abstract: Seventeen students completed a course in which no face-to-face meetings and no paper exchanged hand. All information was shared either on the WWW or by email. In the first few weeks, extensive email dialogue occurred about the method of learning but after that the students focused exclusively on the content of the course. The first readings and exercises gave the students much freedom of choice but that proved too much freedom for the students. They preferred the assignment to be highly focused. The automatic indexing of email by the Hypermail program needed to be augmented with manual indexing by the teacher. This indexing and generally managing the email was the most time consuming aspect of the teaching, but in the end this was less demanding on the teacher than would have been a traditional course. The students rated the course as an above-average learning experience. Keywords: Internet, educational applications, groupware, human-computer interaction, distance education, WWW, email, classroom 1 Introduction
The information superhighway is helping change the education marketplace [Rada et al 96]. Many people accept this statement but many debate its particulars [Massy and Zemsky 95]. The traditional goals of higher education are to improve the quality of education, to increase access for students, and to lower the per student costs. What is the impact of technology on these factors [Ehrmann 95]? Many experiments in technology and education have looked at the impact on quality. For instance, researchers have studied the impact of collaborative learning supported by networked computers in the classroom and seen benefits to student-student interaction mediated by the computer [Rushton et al 93]. The information superhighway can reach people who would otherwise find access to education too costly. Education at a distance has been occurring for many years and can be managed, for instance, entirely by paper mail. The information superhighway allows us to engage new experiences in geographically distributed learning. Various institutions have been teaching extensively with the internet for years. For instance, at New Jersey Institute of Technology degrees are available for students who work at a distance [NJIT 96]. The teachers have found that a combination of Page 732 videotaped lectures and internet interaction works better than internet alone [Hiltz 95]. Hiltz has used a proprietary internet application to support her educational program. Positive results have been presented with various specialized learning programs that operate across computer networks [Rada et al 93], [Brailsford and Davies 95]. We rely exclusively on the WWW and email support for the effort. We do use some public domain software applications atop the WWW but nothing else. We then ask how should a teacher manage such an educational set-up and how will the students respond. We are not inventing new technology. The simple application of existing technology is often better than the complex application of new technology when moving an organization into virtual mode [Rada 95a]. This generalization should apply to moving a course into virtual mode. We present the results of one particular course in one university. We only indirectly have inferences to make about entire curricula. 2 The Course Main Attributes
Our course was titled "The Virtual University" and classified as a computer science graduate course. It was taught at Washington State University in the spring of 1996 [Rada 96]. To enroll students needed to demonstrate a familiarity with the internet but there were no other prerequisites. Lectures were not held but instead students communicated with one another and the teacher on the internet. All course reading material was available on the WWW and all student work was submitted on the internet. This was the first offering of this course under this title and the first time this teacher (or these students) had participated in a course without face-to-face scheduled meetings. The teacher has been for 10 years teaching courses in which all reading material and exercises were stored on a computer network, and students submitted their work that way -- but those courses did have scheduled meetings. The logical progression of topics was from definition of a virtual university to tools and methods in virtual universities and finally to small and large applications in the public and private sector. Students were also expected to work in teams on substantial projects. The intention in such a course is that the professor operates as the "guide on the side" rather than the "sage on the stage". The professor should guide the students to a library of reading material and suggest exercises. The students should select the reading of greatest interest to them, do the exercises, and extensively communicate with one another electronically. The professor's major responsibility should become that of mediating the discussion so that it leads to the greatest insights for the students. Fifteen students were enrolled in the course. They held various positions in the university: Page 733
All the students held their major position at the Washington State University main campus in Pullman. The home page of the Virtual University course was dominated by three images: one pointed to the classroom, one to the teacher, and one to the library for the course [see Figure 1].
Figure 1: This is the largest part of the Home Page of the Virtual University Course [Rada 96]. Each of these icons could be selected to take the reader to the appropriate section of the course. Each of the three icons corresponded to an important source of course information:
The "Library" Icon pointed to 15 topics. Each topic was associated with reading material and an assignment. The course information was updated on a weekly basis but the home page remained basically the same. 3 First Weeks
The first weeks of the course were very different administratively speaking from the later weeks of the 15-week course. At the beginning, much of the attention was on the method and tools of the course rather than on the content itself. The teacher also learned through the first few weeks that the students appreciated more focus in their assignments than he first expected. 3.1 Coordination Focus
From the students' perspective, the course beginning was dominated by needing to understand how such a course would run. In the first weeks of the course the email discussion was often about the medium and method of the course. We have summarized the email discussion (recall the class discussion was essentially by email) by categorizing email into two broad categories of:
Within the Coordination Topic we further divided the email into several subtopics as follows:
The number of emails in each subtopic by frequency indicates the greatest frequency (18) occurs in the subtopic of "Methods" [see Figure 2]. If one groups the subtopics on "Technical" issues, then their frequency is 12. Thus, how the class should run and how to use the tools are the most frequent concerns (30 of 48 messages) of the students within the broad "Coordination" topic.
Figure 2: This graph shows the frequency of email on each of the 6 Coordination subtopics. In the first weeks of class, two students expended considerable effort in trying to arrange face-to-face meetings with the other students in the course. The professor and these two students did meet twice and invited all other students to join, but no other students came to these arranged meetings. In the later part of the semester no further efforts were made to arrange face-to-face meetings of the students. In the last weeks of the course administrative discussion largely disappeared, and the focus of the email was on the content of the course. This is made clear by the chart of "Coordination" email by week [see Figure 3]. All but one of the "Coordination" topic discussions occurred in the first 5 weeks of the course. Page 736
Figure 3: This chart shows the frequency of email from students on each of the Coordination subtopics by week. No Coordination email from students was recorded after week 9. "Read Tech Iss" is short for "Reading Technology Issues". "Wk n" is short for "Week n". 3.2 Content Focus
The teacher had initially created weekly assignments that were based on a largish collection of WWW readings. The teacher assumed that students would pick and choose from the readings those that they found most interesting. The assignment each week was to then write a short essay to reflect what the student had learned in the readings. For example, the first week library pointed to the following 4 readings (directly copied from the course WWW site):
Page 737 The student was expected to write a short essay about what he/she had learned relative to the reading and the topic of the week. However, students were confused with the large selection of reading material. The students and teacher agreed on a modified approach in which each week's assignment included a small set of required, primary reading and a larger set of optional, secondary reading. For instance, the topic of Week 10 was "Coordination," and the presentation on the WWW for that week lists
The primary reading asks the student to "Visit two sites that describe first a WWW conferencing system and then a WWW workflow management system.." It continues by pointing to two specific documents, one about San Francisco State's Web Conferencing System and the other about Action Technologies Workflow Management Coalition. The assignment then said: "Compare and contrast the features of the Conferencing on the Web system with the Workflow system. Then suggest how each system might be used in our class. What would be the pros and cons for our class of using either or both of these systems?" The optional reading pointed to four other sites for further, related materials. Students much prefered the focused reading and assignment of the 10 th week over the relatively diffuse reading and assignment of the 1 st week. 4 Mid-term
In the middle of the course, the level and nature of the activity changed. Students had sensed the freedom of the course and were not reliably doing their homeworks punctually. Also the amount of email that had been received and archived at the course address had grown to such an extent that the initial simple indexing scheme was no longer adequate. 4.1 Enforcing Pace
The difficulty for the teacher revolved around the monitoring of the student work as reflected in the electronic mail. Traditional courses involve physical meetings and a clear sense of pace. The teacher only applied gentle pressure to students to complete their submissions within fixed weekly time frames, and many students took advantage of this freedom to make their contributions at varying intervals after (or at times before) the announced due date. Page 738 In the first two weeks most students submitted their homework on time [see Figure 4]. In the middle of the course this was less often the case. Near the end of the course students appreciated better that all work had to be submitted in order to get as much credit as possible and they then submitted work that might have been overdue by weeks. The teacher did not enforce strictly a penalty for lateness and this approach contributed to the tardiness of the submissions. A teacher could enforce stiff penalties on late submitters; the impact of such an approach should be tested in future runnings of such a course. Further explanation for the data in the Figure 4 addresses the blips and the lowering numbering of submissions per topic. The blips in the 5 th , 10 th , and 15 th week are partly due to the heightened importance of those works to the student grade. The reduced total number of submissions is due to the emphasis on submitting work by team and the formation of some teams only later in the semester (and subsequently submitting just one work per team).
Page 739 4.2 Archiving Discussion Meaningfully
Students were required to submit their exercise solutions to the course via email. The email address was connected to a program that archived the mail on the WWW. The program to do this is called Hypermail [EIT 95] and has been found useful in other collaborative projects [Cutkosky et al 96]. However, the automatic archiving by Hypermail did not seem to provide enough structure to focus the attention of the course participants on timely issues as they arose. Hypermail automatically indexes email by its sender-assigned subject, the userid of the sender, and the date. As the subject headings that senders used were not rigorously controlled, the Hypermail index proved largely useful as a date and author index but not as a subject index. The teacher broadcasted messages to the students every few days. These transmissions collected the comments of the teacher on the student submissions since the previous teacher broadcast. Organizing these teacher comments and the student submissions so that they corresponded with the weekly list of topics was an additional maintenance task for the teacher. Such maintenance work might be done effectively by a teaching assistant. Also more sophisticated computing environments could automate some of this maintenance effort. 5 Final Evaluation
In the end, what did the students think of the course? Did the teacher want to repeat the experience in another course? 5.1 Student Perception
The standard university evaluation form by which students rate teachers was placed onto the WWW. Students answered anonymously. Seven students in the course completed this optional course evaluation. The questions on the evaluation ask the student to rate on a scale from 1 to 5 each of 26 different attributes of the course. 5 means an excellent course in that attribute whereas 1 means a very poor course. Across all answers and the 7 students, the average reply was 4.63. The average score across courses in the school is much closer to 4. The three questions scoring highest were:
The three lowest scoring questions were:
with scores of 4.00, 4.29, and 4.29, respectively. Overall, the students were satisfied with the course. 5.2 Teacher Perception
The teacher felt that the students had learned. The student grades were high as is usual for a graduate course in computer science, with the average being a low 4 on a 1 to 5 scale (where 5 is best). In terms of teacher time, the initial weeks of the course were more time consuming than other courses might have been. The teacher needed to design the WWW format for the course and develop the syllabus and reading materials. Furthermore, as the course started and students had many questions about the method, the time spent in both covering these method questions and dealing with the content was enough to make the teacher believe that a traditionally taught course would have been less demanding of the teacher's time. However, after the 5 th week the questions about method from the students stopped. Also by then the WWW site had stabilized and the teacher found that the course required significantly less time than a traditional course. In the last part of the semester the most time consuming task for the teacher was
Efforts to get students to do the structuring and the assessing themselves have met with mixed results over the years [Rada 95b]. While tools could be built to facilitate the semiautomatic structuring of student submissions and student-student self assessment, the student naturally expects the expert teacher to do some of this. In the traditional university lecture, the teacher may have relatively little interaction with students but the students might still feel that their work is somehow well in step with that of the teacher. In contrast a no-lecture course requires other methods of generating a sense of course cohesiveness. In the end, whether or not a course costs more or less for the teacher to give on the WWW versus in a lecture hall must depend Page 741 on many specific factors of the teacher, the students, and the way the teacher and students interact. 6 Conclusion
The course went through at least two distinct phases. In the beginning the students were very much concerned with how such a course would operate. The teacher was extremely busy both assuring the students that they could learn without attending lectures and providing them a worthwhile learning experience. Later the students' concern with method disappeared and the teacher spent less time on the course. In the beginning, the teacher was lax in enforcing late submission penalties. This led some students to submit their work very late and to lose some of the pace that is important in a course. Many students worked full time and could not practically go to lectures at fixed times during the day. The course offered access to these students in a way that traditional lecture courses would not have. This is the primary benefit of a course taught over the internet that does not require any face-to-face meetings. Students could have easily participated from anywhere in the world. The toolset used for the course was basically the WWW and email. The WWW tools included forms for collecting input on the course evaluation. Hypermail was one add-on that proved enormously helpful. We were not attempting to be minimalist in our use of software but felt the main challenge was the organization of information and the interaction among participants. In a future edition of this course the teacher would like several changes. On the technical side, the teacher would like a database management system connected to the WWW and functions to further automate the tracking of student performance as reflected in the records of the database. On the administrative side, the teacher would like to have more students enrolled from distant places -- students who truly could not come to lectures in one town. This course had already proven a success under the conditions in which it was offered. New markets of students could be reached with this virtual mode for educational delivery. The methods and tools are not difficult to fashion. The challenge is to put together the appropriate group of students, teachers, and administrators. References
[Brailsford and Davies 95] T J Brailsford and P.M. Davies "Collaborative Learning on Networks", CAL '95, Cambridge, April 1995 http://ibis.nott.ac.uk/cal95/ktpaper.html (1995)
Page 742 [Cutkosky et al 96] Mark R. Cutkosky, Jay M.Tennebaum, Jay Glicksman, "Madefast: an exercise in collaborative engineering over the Internet" Communications of the ACM, Sept. http://www.madefast.org/mf/ACM_paper.html (1996) [Ehrmann 95] Stephen Ehrmann, "Asking the Right Questions: What Does Research Tell Us About Technology and Higher Learning?" Change: The Magazine of Higher Learning, XXVII: 2 pp 20-27, March/April (1995) [EIT 95] EIT "Hypermail" http://www.eit.com/software/hypermail/hypermail.html (1995) [Hiltz 95] Starr Roxanne Hiltz "Impacts of college-level courses via Asynchronous Learning Networks: Focus on Students" Sloan Conference on Asynchronous Learning Networks, Philadelphia, October 1995, http://it.njit.edu/njIT/Department/CCCC/VC/Papers/Async_Learn/Philly.html (1995) [Massy and Zemsky 95] William F. Massy and Robert Zemsky "Using IT to Enhance Academic Productivity" Interuniversity Communications Council, Inc., http://educom.edu/program/nlii/keydocs/massy.html, (1995) [NJIT 96] New Jersey Institute of Technology (NJIT), "BA in Information Systems via Distance Learning", http://www.njit.edu/njIT/instruct/degree/infosys.html, (1996) [Rada et al 93] Roy Rada, Sharon Acquah, Beverly Baker, Phillip Ramsey "Collaborative Learning and the MUCH System", Computers and Education, 20, 3, pp. 225-233, (1993) [Rada 95a] "Interactive Media" Springer-Verlag, New York (1995) [Rada 95b] "Developing Educational Hypermedia: Coordination and Reuse" Ablex Publishing, Norwood, New Jersey (1995) [Rada 96] Roy Rada "Virtual University Course" taught spring 1996 at Washington State University as Computer Science 580 and available at http://www.eecs.wsu.edu/~virtual/ (1996) [Rada et al 96] Roy Rada, Jim Rimpau, Curtis Bowman, Jerry Gordon, Tom Henderson, Todd Sansom "WWW Activity and the Virtual University" Educational Technology and Management to appear (1996) [Rushton et al 93] Chris Rushton, Phillip Ramsey, Roy Rada "Peer Assessment in a Collaborative Hypermedia Environment: A Case Study", Journal of Computer-Based Instruction, 20, 3, pp. 75-80, (1993)
Page 743
|
|||||||||||||||||
