Last changed 13 Jul 1998 ............... Length about 1000 words (7,000 bytes).
This is a WWW document maintained by Steve Draper, installed at http://www.psy.gla.ac.uk/~steve/overview.html. You may copy it. How to refer to it.

[ Overview page    Teachers' main page    Learners' main page    Sample home page for a delivery ]

CBL evaluation: Overview of the ATOM

Summary

State the topic and type of ATOM.
This ATOM is a practical introduction to evaluating CBL material. It was intended as one week's work on a CBL course, but is probably rather more than that: one of the bigger exercises on a course. It spreads over several weeks in elapsed time because of the need to plan, execute, and analyse an evaluation. (The sequence of activities is given on the learners' main page.) Timetabling may be constrained to fit around the delivery time of the CBL chosen to be evaluated.

How the concept fits into the course

Relationships and constraints on this topic in relation to the surrounding course
This ATOM was designed for use in a module on CBL (computer based learning). Evaluation, though often neglected, is a crucial topic. Conceptually, it could be taught at any point: you could begin with it to set a standard, or end with it. The kind of evaluation presented here is empirical, classroom based evaluation i.e. it is about observing and measuring student reactions and learning outcomes of CBL (not about software features, or teacher opinions).
Student prerequisite knowledge
None, although it is assumed that the students will have some knowledge of HCI evaluation or else they may be unready to accept the importance of testing real users.
Student prerequisite skills
It is assumed, although perhaps not wholly essential, that the students will have some experience of HCI evaluation methods, some experience of designing and using questionnaires, and some experience of receiving CBL material themselves.
Course level
This ATOM has been used at M.Sc. level but there is no obvious reason not to use it at any level, except perhaps for issues of independence and study habits. These often depend more upon discipline (arts vs. science) than course level.

However planning and executing an evaluation as an autonomous activity, without simpler warmup activities, probably does require previous experience that may not be available except at this level.

Timetabling: how the activity fits into the course

Amount of student work
Intended as one week's work for a module (say 8 hours), but probably more.
Amount of contact hours
We scheduled 2 2-hour class meetings (using video conference), where this ATOM was a main item of business.
Relationship to rest of course
The key constraint is that this exercise is about evaluating some (other) piece of CBL. When this has been decided on, then the work for this ATOM is scheduled around it: with data gathering during that delivery, analysis and writeup after, and planning before. Thus this ATOM can probably not be the last activity in a course, and the CBL delivery used as a target cannot be the first thing in a course. More details on the sequence of activities is given elsewhere.

Resources needed

Special
The ATOM requires that the students have access to either themselves or another group of students while they are receiving some CBL material.
Remote expert
Used for seminar/tutorial (by video conference) twice: to discuss the reading and/or preliminary design of the evaluation, then to discuss preliminary findings by the student groups. Only 2-4 hours of expert's time.
Software
None (other than that CBL material).
Hardware
None for computing.
Video conferencing for the whole class, if the remote expert is used.
Reading
Papers are available on the web and in journals.

History and admin.

Authored and maintained by Steve Draper at the University of Glasgow.

Number of times delivered: 1
Evaluation reports available: 1
Number of Trails (examples of past student work): none.

[ Overview page    Teachers' main page    Learners' main page    Sample home page for a delivery ]