Last changed 18 Apr 2001 ............... Length about 1,500 words (10,000 bytes).
This is a WWW document maintained by Steve Draper, installed at http://www.psy.gla.ac.uk/~steve/grumps/protocol.html. You may copy it. How to refer to it.

Web site logical path: [www.psy.gla.ac.uk] [~steve] [grumps] [this page]

GRUMPS computing lab study: Participants' view

by Steve Draper

Contents (click to jump to a section)

Preface

This outlines the startup study that the GRUMPS research group plans to do in computing science lab classes. The document is addressed in particular to those staff responsible for those classes, and whose permission and cooperation is essential for this study to proceed. It is meant to describe what it will mean to, and how it may impact, participants; but not to describe the research issues and why these might be interesting (which however we are happy to discuss at tedious length if asked).

However this version of this document is not fully up to date and may have some inaccurate details.

Introduction

This project is mainly a pilot study, aimed at testing the feasibility of various ideas and uncovering unforeseen issues with them. It has some distributed systems, some HCI, and some educational research aims. If approved and ready in time, it will begin on day 1 of term 3 (i.e. Tuesday 17 April) in a first year computing lab.

A companion document outlines the structure of the software, the likely "safety" issues and testing from the viewpoint of maintaining smooth running of the lab hardware and software, and the disaster management plans, precautions, and facilities. This document focusses on what the study involves for humans: staff, students, and researchers.

The functions

The study will introduce several functions.
  1. User input recording: a facility capable of collecting all user input and storing it in a database. This is designed to be completely ignorable by students, but will have a visible control allowing the student to turn it off, or to restrict the amount of data captured.
    This explores technical issues of distributed information management; HCI issues of privacy for the users; and potential for educational benefit (yet to be demonstrated) in revealing informative patterns over time in the students' activity.
    1b) User log summaries: Offering some information back to students on their own patterns (i.e. after 2 weeks, displaying the results of some simple queries on this data). This will be a first exploration of what learners find interesting about their own activity data, what kinds of thing could be educationally useful in any way, and also relates to the privacy issue by making more clearly visible the kind of thing that has been recorded.

  2. Lab Support Software (LSS): featuring a help queue mechanism, allowing students to signal to the tutors and demonstrators that they wish for help, and also to indicate the topic they want help about. The queue, held on a small database, will be visible by all in the group (students, tutor, and demonstrator) as web pages, and will use web forms for user input. The information may include: the set of students, their location in the lab, whether and when they logged in (and out), asked for help, last got help. This should support tutors scheduling the order in which they see students by any of these attributes (e.g. seeing all students in turn, probably ordered by physical location; and at other times, by who has been waiting longest); allow students to signal for attention without having to keep their hands up for long periods; and allow students to notice that someone else has asked about a topic and indicate that they too want to hear about that. It may also turn out to be helpful to tutors administratively: in effect keeping a register automatically, and making it easier for tutors to check whether there are students they haven't spoken to for a long time.
    The mechanism can always be ignored without cost by any group at any time (the alternative of students raising their hands is always available): no doubt this will be controlled by simple announcements by the tutors.

  3. Tutor handheld interface. Exploring the use of handheld computing devices (e.g. PDAs) for tutors displaying the help queue (again as web pages), possibly with additional information not available on the pages for students, and allowing input validated as from the tutors. (Only) some of the useful extra information is lost by the tutors if these fail to work, or prove insufficiently convenient.

  4. Not implemented in this startup study, but aimed for in future developments, is an additional facility for tutors to invent a "milestone" set of categories for expressing student progress on the current lab task, and entering or updating this information about each student (presumably having just seen them). This could be an additional criterion for choosing which student to talk to next, as well as helping tutors to keep track of the progress of each student in their group.

Timetable

Students' viewpoint

Students will be given a written explanation of the study (preview by email, personal paper copy on arrival in the lab), plus a consent form to sign. Whether or not they sign, they can turn the data recording off (or back on) personally at any time on their machine by the control panel.

The control panel makes it open and visible that recording is taking place, and is an active reminder of it. The levels offered (possibly described in different words) will be:

Students will be able to request from us a complete listing of all data about them kept (i.e. complete log of their input data).

We will ask a sample of students to complete a short questionnaire in the first week, and again in a later week. We propose to do this in two of the tutorial groups (subject to tutor approval).

We will also ask a smaller number of students, on a paid volunteer basis, to maintain simple "diary" records during the labs. (Main purpose: get some data on how long students wait to signal without the new software before putting their hands up, and so get a measure of the real waiting times.)

We plan a debriefing event before the end of term, to ask for any comments and discussion from the class reps to the project team: both to maintain good relationships with the students and to hear any reactions or comments.

Lab tutoring staff viewpoint

When the LSS is turned on (week 2), they may get their group to use the electronic queueing system, though they may abandon it or respond to a mixture of the electronic and traditional "hands up" signals. We envisage the queue showing who has requested help, not imposing priorities on whom the tutors serve first.

Those offered the handhelds (e.g. PDAs) may find them more convenient as their interface to the queue, and will allow fuller data to be captured about the start time (and hence duration) of each visit to a student which itself may help make the queue more useful. Of course they may choose not to use/do this.

Staff will be asked to fill in a short questionnaire on their lab strategies early on, and afterwards for their comments on any changes in strategy prompted or supported by the software.

We will also have an observer in the lab periodically in addition to project personnel there to manage the handheld equipment and as a precaution against software problems.

We plan a debriefing event before the end of term, to ask for any comments and discussion from the staff to the project team: to maintain good relationships and to hear any reactions, comments, insights, and suggestions for improving the designs.

Web site logical path: [www.psy.gla.ac.uk] [~steve] [grumps] [this page]
[Top of this page]