A Monitoring Program for Evaluation of Educational Software on the World Wide Web PROCEEDINGS
Edna Ruckhaus, Universidad Simón Bolívar, Venezuela
EdMedia: World Conference on Educational Media and Technology, in Seattle, WA USA ISBN 978-1-880094-35-8 Publisher: Association for the Advancement of Computing in Education (AACE)
The evaluation of educational software can be divided into two phases: 1.- the formative evaluation which is done
during software design and development, its aim is to determine the capability of the software to satisfy user learning needs,
and 2.- the summative evaluation which is done once the software is implemented, its purpose is to relate the context of
usage to learning effectiveness. In both phases, educators and software developers have relied on traditional methods of
evaluation like interviews, direct observation, pre-tests and post-tests. During the past few years, the number of educational
applications that is published on the World Wide Web has increased considerably, meaning that we can no more evaluatethe
usage of software in a controlled environment.
The purpose of this project is the development of a monitoring program that automatically generates information that is
useful for carrying out summative evaluation on educational software that is used on the World Wide Web. The program
collects data (measures) on a group of variables, several of the results are expressed in terms of fixed parameters that
represent the logical or recommended student performance. The monitoring program stores the measures in a database: raw
data can be consulted, and statistics are produced on demand. In this way, the results are available for analysis and
Experts in design and usage of educational software introduce the parameters. The final comparison and analysis
should be done by experts in content, usage, and educational techniques, technical and graphical experts, and language
communication experts. An important part of the project has been the development o a framework for analysis that relates
measures to software characteristics: these include flexibility, legibility, consistency, grammatical correctness, interface,
content, degree of interaction, browsing capabilities, media resources and technical features. For each characteristic there is
a set of indicators of possible software flaws.
The implementation of variables has been done for a prototype tutorial course on Local Area Networks that is
representative of a hybrid educational application (tutorial & drill). The course was developed in HTML. The Variables
4GL CGI (Common Gateway Interface). The next research phase will be the analysis of data collected by the monitoring
program during its usage by a group of Systems Engineering Students. Simultaneously, data will be collected and analyzed
through more traditional methods. By comparing both analysis, the monitoring program will be validated.
Ruckhaus, E. (1999). A Monitoring Program for Evaluation of Educational Software on the World Wide Web. In B. Collis & R. Oliver (Eds.), Proceedings of EdMedia: World Conference on Educational Media and Technology 1999 (pp. 1402-1403). Association for the Advancement of Computing in Education (AACE).
© 1999 AACE