Web site logical path: [ www.psy.gla.ac.uk] [~steve] [mant] [this page]

Summative project evaluation

This report is assembled mainly from other papers in order to provide a deliverable promised in the MANTCHI grant application, where it is described only as "Summative project evaluation" (item number 15). The meaning and scope of that now seems ambiguous between:

1. The evaluation work we did. Part B presents a large overview paper on the (integrative) evaluation work done on the project, which corresponds to reporting on what we did for project objective 2 (of 4): "To measure the educational effectiveness of this novel delivery of tutorial support using the method of Integrative Evaluation". One of the products of this evaluation work was a set of recommendations for delivering our teaching materials: these are presented in part C.
2. Evaluating our evaluation work. Part B also contains our self-criticisms of our evaluation method. We did what we promised, but what with hindsight might have been better evaluation aims?
3. Evaluating the project, not the educational work. This should surely be the content, not of one deliverable from part of the project team, but of the project final report. However as the project evolved and we perceived new needs, we did perform a study of a different kind which is important to an evaluation of the project itself: the cost benefit analysis of using our approach to collaborative tutorial teaching. This analysis is reproduced as part D. In addition, the conclusion (part E) offers a brief summary of the features of the project as a whole, its results, and their relationship of the evaluation studies.

This report therefore consists of:

• Part A: This introduction.
• Part B: a long overview of the project's integrative evaluation work, based on 20 studies (listed in appendix 1). One output of this is part C.
• Part C: a list of the recommendations for delivering the ATOMs based on the evaluation studies.
• Part D: a cost benefit analysis of the MANTCHI approach to remote collaborative tutorial teaching based on an interview study of the teachers involved.
• Part E: A short overview of the relationship of these studies with the project as a whole, and its main aims, themes, and innovations.
• Appendix 1 The list of evaluation studies performed
• Appendix 2 An example questionnaire instrument used

Status

The document contains a table (in WORD); and appendix 2 contains some Zapf font characters (boxes).