Main text last changed 21 Apr 2012.............. Length: 2,800 words (20,000 bytes) .
(Document started on 27 Dec 2009.) This is a WWW document maintained by Steve Draper, installed at http://www.psy.gla.ac.uk/~steve/rap/fcal/main.php. You may copy it. How to refer to it.

Web site logical path: [www.psy.gla.ac.uk] [~steve] [rap] [this page] [print compilation]
[localed] [innovations] [this page] [Table Restructuring tool]

Publishing feedback calendars

By Steve Draper, Department of Psychology, University of Glasgow.

The main idea

The idea is for a course team to publish the timetable of feedback that students will receive: when and of what types. Entailed by this will be the timetable of assessments and hand-in dates too.

If you want to try it yourself, to rough out what one for your course might look like, an easy way to get started is to download one of the templates in Word and start editing.

Examples being developed in 2010-11

L4 psychology [HTML]
L3sem1psy [HTML]
JimBaxter [HTML]
L1 psychology [HTML]
CS1P [HTML]
BodyBelief [HTML]
Plain Template v.1 [HTML Word]
Annotated Template v.1 [HTML Word]

Examples in use or development at GU in 2011-12

Philosophy: Level 1, semesters 1 and 2 (approximately 400 students). Tutors personalised a generic feedback calendar, and posted it onto their forum. Class login required to see this. One example.

Psychology: feedback calendars incorporated into course documentation for Level 1, Level 2, and Level 3 (separate document) courses.

Music: feedback calendars incorporated into Levels 1 and 2 (Music runs classes across levels and year groups). Course names: Musical Culture in the Long Nineteenth Century; and Musical Techniques, Intermediate, (Level 2/3?). http://arts.moodle.gla.ac.uk/mod/resource/view.php?id=100346 copy.

Physics: feedback calendars have been drafted but not yet finally approved for 3 courses: P1 (N = 250), P3 (N = 90), EE1physics (N = 40). Contact: Eric Yao.

Computing Science: feedback calendar created, and distributed in week 7. Contact: Quintin Cutts. copy.

Who was interested but didn't adopt them

xxx

Candidate columns and their meaning

Dates in general Dates given are often indicative only, not precise, for numerous reasons e.g. if it happens at a tutorial, it will be on a different day for different groups; just because it is "available" on a given day does not mean the student gets it that day: and no learning occurs until the student does receive it so the student-centered not the administration-centered view is what is relevant. For illustrative purposes here I have not worried about precision. The design question is: would such calendars be more useful if a single indicative date were replaced by a range of dates or complex lists of the date as it applies to each student?
Unseen exams will have the first and second dates identical. On the spot quiz marking, and EVS, will have the second and third dates identical.
Work start date The start date can be definite when essay titles or problems are handed out; but is often over a range where students create or select their own topic; or may be of little relevance if a writeup depends on when the last data are collected.
Work deadline (date)
Feedback available (date) Besides "available" not being the same as "received", oral and written aspects may become available at different times.
Expert / peer Are the marks and/or comments generated by tutors or peers; or more precisely, by an expert or a colleague relative to the learner. (Whether the "tutor" was the course leader or a teaching assistant, a professor or a student in the year above, is assumed unimportant here.)
Mark (A learner may receive marks or comments or both.) Here "mark" means "summative success metric" i.e. whether learners get information on right/wrong and degree of success. A separate meaning, not recorded here (it will undoubtedly be presented in other documentation), is whether this affects later summed marks for certification. A feedback calendar is NOT a table of rules affecting certification.
Comments I.e. "feedback" as opposed to marks, formative information as opposed to summative. I've estimated the average amount: in words for written comments, in minutes of speech for oral comments. Often, feedback is partly "generic": comments applying to the class as a whole, not individualised. This is often helpful to students, who thus get to see issues they weren't fully aware were as important as they are.
Written / oral If oral then assumed to be face to face. Of course, comments may be both i.e. tutor hands the learner written comments but then elaborates on them. Marks are also deemed "written" if delivered via mark databases, lists, etc.
Discussion / Dialogue The issue here is whether there is dialogue, 2-way or multi-way discussion about the feedback. Discussion with peers is often useful as well as with experts. In many cases dialogue is offered but not structured into the course, resulting in a few percent of the class taking up the offer. Coded in the tables here as "normally no". Obviously, as chat with friends vs. consulting a physician illustrates, discussion is much more likely if it does not depend on making an appointment. An appointment system saves a lot of staff time; but almost certainly fails to deliver a lot of potential benefit. It means that only students with specific and confident questions get discussion. However, much of what we learn comes from dialogue other than us asking questions: otherwise we wouldn't have lectures, read newspapers, etc.
Solo / group communication There are really two issues: a) Are the comments/marks personalised to each student OR are they generic for the group as a whole. This first issue is indicated in the "comments" column by whether those are labelled "generic". b) Are they communicated in a group, or individually so that students hear nothing that applies to other students. Whether or not they are always comfortable about it, students learn from comments directed at other students or the group as a whole. And if delivered in a group setting, the chances for discussion, especially including peer discussion, is obviously much greater. Some students get together to compare their marks and feedback, in order to get this extra information.
Alternative good answers shown Contrary to commonsense meanings of "feedback", students actually gain a lot, perhaps even more, from seeing other ways of answering an essay or problem than from the judgements of the marker. Only after a learner knows more than one approach -- and so has a choice -- does judgement of which one is best or right become relevant. This needn't mean having examples shown in advance, but having students read each others' work afterwards. One approach is model answers. But in some ways, real student answers are more informative. This certainly applies to essays; but science students can be even more surprised to realise that there were quite different solution approaches to a problem than the one the came up with.

The second idea: timeline diagrams

The first idea for feedback calendars was to show tables of feedback occasions and their types (their attributes), including dates. A second idea is additionally to represent the same occasions but in relation to a timeline. Each chain shows an occasion and the work that precedes it by student and staff. Chains show the sequence, and can show a few fixed times such as handin deadlines; but more often, the period during which actions must be fitted in.

By showing all the chains involved in a course, one can see student work as a whole; by showing all the chains involved in all the courses a student does in one period, their whole workload can be overviewed.

A few diagrammatic examples of this are (will be) here:

Alternatively, chains can be represented textually as bullet lists or tables:

An essay assignment
- < Start date 15 Sept. Essay topic announced.>
- Students: read texts, write notes, write drafts.
- Students: final draft
- < Handin date 15 Nov.>
- Staff: Marking
- < Hand back date 30 Nov.>
- .......
- < Start date 15 Jan. Semester 2 essay topic announced.>
- Students: review feedback from previous essay.
- Students: read texts, write notes, write drafts....

Evidence

None: no-one has tried this yet as far as I know, although related interventions are currently (2009-10) being trialled (see "related ideas" section below).

What it can accomplish

A written calendar in tabular form, by providing an external visual representation of aspects of the whole course, naturally prompts reflection on it as a whole. Many of the benefits mentioned below flow from this. The aspects represented (e.g. by a column in the table) determine what gets reflected on. This, in different ways, works on both staff and students.

In fact its main broad benefit is to make both staff and students more proactive about feedback; and this simple indirect method of prompting reflection by an explicit representation may be the first and most important step in this.

By being made aware of feedback occasions whenever they look at the calendar (rather than as an incident that only lasts a few minutes and might be overlooked), students are more likely to recognise it when it occurs, and to recall it when answering the NSS.
By being made aware of feedback occasions, students are more likely to perceive the place and role of feedback in the course.
It expresses the idea that feedback is a significant part of how the course is supposed to work in promoting learning. This is often left entirely unstated, and both staff and students forget it, and go through the motions without making any connection to its supposed meaning and function (like Christmas, or saying "thank you").
By being made aware of feedback occasions, students are perhaps more likely to become more active in exploiting the opportunities e.g. by asking questions to get more information, by considering how to use feedback in their work. (This has certainly happened as an effect of related interventions.)
By representing a summary, the calendar can prompt review of the types of feedback provided, depending on the columns used. For instance, the balance of tutor and peer feedback; the provision of opportunities for discussing feedback as opposed to delivering it like a tax assessment (i.e. judgmental, hard to understand, and very hard to find someone to explain it).
It could also represent not only fixed events but suggestions e.g. for students to organise peer groups to compare and discuss the feedback they receive. See here.
It also contains a calendar of assessment: and so notifies students of hand-in dates. And not as isolated announcements, but in a form helpful for planning their work throughout the course and across courses.
Ditto for staff about their marking duties, notifying staff about planned turnround times.
As such, it is also a useful first step in addressing how course design is not just a design for staff duties, but is also in fact a design for students' out of class work; and in reminding the staff to think about whether this is spaced out feasibly from the students' viewpoint.
And a useful first step for coordination across courses, addressing the common problem of all the work being demanded simultaneously in all courses. Putting the calendar on the web allows other departments at least to inspect the dates, before any coordination procedure has been worked out. And of course it draws student attention to such timing problems earlier and more clearly, even if staff fail to inform themselves about this.

Related ideas

As soon as I heard about various recent innovations, feedback calendars seemed an obvious development of them in the light of current ideas on assessment and feedback.

The closest precursor was a two (rather than 11) column table of "module feedback opportunities" on "individual projects", a component of a course at Strathclyde University.

Feedback week: announce across the first year, the week when students get their first feedback from all courses. (Durham: see here.)

Announce a feedback fair or "feedback intensive": e.g. a half day event when students can drop in to discuss any or all feedback, and perhaps with other feedback related features: posters?, PAL sessions for discussing feedback, ...

Assessment-only calendar: (a.k.a. assessment diaries) for coordinating hand-in dates across departments and courses. (Also useful within departments to manage marking loads where some staff must mark in multiple years/levels.)

[Frank Coton's method.] Hold a series of staff-student meetings on the meaning and role of feedback, to communicate what it means to each other. On the one hand, students might convey to staff how pointless feedback seems when what they want to know is are they capable of continuing in this subject or are their aptitudes better suited to switching to a different subject; how labs appear places of required attendance (like meeting your probation officer) and demonstrators there to tick you off a list like a supermarket checkout operator. On the other hand, staff might convey that the provision of lab. tutors and demonstrators is one of the most expensive provisions of feedback and assistance opportunities that the course has, whose main justification is answering personal questions put by students. It is reported that this not only changes student perceptions (and so their responses to the NSS), but more importantly starts to change their behaviour and make them more proactive in taking up these opportunities.

Relabelling tutorials: (psych. dept., ) Simply renaming tutorials as "feedback / feedforward sessions" can refocus students (and staff).

Student negotiation: (Engineering, ) Pre-agree before the course starts, through negotiations between the course team and class reps: what will the the type and timing of the feedback be. I.e. decide on the (calendar) dates.

Criticisms

There are important features of feedback practice that can't easily be read from these tables. One is: weaving together negotiating marking criteria and then doing the exercises; another is how (if) feedback from one activity can be used directly in another (e.g. if a talk is assessed and then an essay written on the same topic as the talk).

Another issue is what "marks" mean:

something that motivates students by affecting their final grade for the whole degree or course

a measure of right/wrong-ness (as distinct from linguistic descriptions of what is right / wrong / could be done differently).

What is the audience for a feedback calendar?

The audience is both staff and students; to facilitate each person taking an overview of this aspect of the course, and so prompting reflection on feedback, on how different people's actions feed ito the others', the relationship of staff and student actions; and above all, on what action could a student take (and when) on the basis of feedback they receive. This note may not altogether belong here, but is perhaps relevant to starting to apply feedback calendars to your own course. Who is the calendar for? Primarily it is the course leaders addressing the students; but of course they are also addressing themselves with a memo partly about what they must remember to do.