How to Evaluate Educational Games: a Systematic Literature Review

Giani Petri (Federal University of Santa Catarina, Brazil)

Christiane Gresse von Wangenheim (Federal University of Santa Catarina, Brazil)

Abstract: Educational games have been used as an innovative instructional strategy in order to achieve learning more effectively. However, it is essential to systematically evaluate such games in order to obtain sound evidence on their impact. Thus, the objective of this article is to present the state of the art on how to systematically evaluate educational games. Therefore, we performed a systematic literature review with an initial sample of 21,291 articles from which 11 relevant articles have been identified, describing 7 approaches to systematically evaluate educational games. Based on these studies we analyze how the approaches are defined (quality factors, theoretical constructs), operationalized (research designs, data collection instruments, data analysis methods), how they have been developed (development methodology) and evaluated (evaluated aspects, number of applications & data points and data analysis methods). As a result, we can confirm that exist few approaches to systematically evaluate educational games. The majority of the approaches are developed in an ad-hoc manner, not providing an explicit definition of the study, its execution and data analysis. We also observed that among the few encountered approaches no clear pattern emerges on which quality factors to evaluate. This shows that there exists a need for research on the definition and operationalization of educational game evaluations in order to obtain more valid and uniform results.

Keywords: educational games, games evaluation, serious games, state of the art, systematic literature review

Categories: L.0.0, L.5.1