| dc.description |
In order for simulations and games to be effective for learning, instructional designers must verify that the underlying computational models being used have an appropriate degree of fidelity to the conceptual models of their real-world counterparts. A simulation/game that provides incorrect feedback is likely to promote misunderstanding and adversely affect learning and transfer. Numerous methods for verifying the accuracy of a computational model exist, but it is generally accepted that no single method is adequate and that multiple methods should be used. The purpose of this study was to propose and test a new method for collecting and analyzing users' interaction data (e.g., choices made, actions taken, results and feedback obtained) to provide quantified evidence that the underlying computational model of a simulation/game represents the conceptual model with sufficient accuracy.
In this study, analysis of patterns in time (APT) was used to compare gameplay results from the Diffusion Simulation Game (DSG) with predictions based on diffusion of innovations theory (DOI). A program was written to automatically play the DSG by analyzing the game state during each turn, seeking patterns of game component attributes that matched optimal strategies based on DOI theory. When the use of optimal strategies did not result in the desired number of successful games, here defined as the threshold of confidence for model verification, further investigation revealed flaws in the computational model. These flaws were incrementally corrected and subsequent gameplay results were analyzed until the threshold of confidence was achieved.
In addition to analysis of patterns in time for model verification (APTMV), other verification methods used included code walkthrough, execution tracing, desk checking, syntax checking, and statistical analysis. The APTMV method was found to be complementary to these other methods, providing quantified evidence of the computational model's degree of accuracy and pinpointing flaws that could be corrected to improve fidelity. The APTMV approach to verification and improvement of computational models is described and compared with other methods, and improvements to the process are proposed. |
|