Bridging Qualitative and Quantitative Approaches in Evaluating the Educational Effectiveness of a Shared Design Memory

Daniela Giordano (Department of Education Graduate Program in Educational Technology Concordia University, Canada)

Abstract: A shared design memory emerging from the contributions of novice designers affords, theoretically, unique opportunities to support individual and organizational learning. Evaluation must take into account the "distributed" nature of the system that becomes realized. The proposed evaluation model is based on a cross-analysis of: the contents of the shared design memory, the quality of the design artifact produced be the designers teams, the characteristics of the student population, and their perceptions of the adequacy and usefulness of the representational formats adopted for the shared memory. Effects being sought are generational changes that indicate that design weaknesses typical of novices are being offset, and that good design practices are diffused and gradually incorporated as new quality standards. Preliminary results of the evaluation of a shared memory for Information Systems design show that shared memory underpinned an emergent quality in the new designs, characterized by increased structuredness, communicability, and attention to the dynamics of interactions in the system being designed. The shared memory was deemed useful and usable by the learners. Findings also clarify the relative merits of some representational formats (links among design cases and reviews attached to design cases) used for conveying design knowledge.

Keywords: evaluation, information systems design, organizational learning, shared design memory

Categories: H.2.1, H.5.1, H.5.3, K.3.1, K.3.2