Last changed
25 Jan 2003 ............... Length about 3,000 words (21,000 bytes).
This is a WWW document maintained by Steve Draper, installed at http://www.psy.gla.ac.uk/~steve/resources/ffeed.html.
You may copy it. How to refer to it.
Web site logical path:
[www.psy.gla.ac.uk]
[~steve]
[resources]
[this page]
Formative feedback to students in levels 3 & 4
by
Stephen W. Draper,
Department of Psychology, University of Glasgow.
Mainly written December 1999.
This web document as a whole is a proposal for supporting an increase in
quality of feedback to our honours students. If accepted, the action would
be: a) a standard prompt sheet circulated e.g. by Anne
Tonner with all the marking in level 3. b) A document available on the web
with justifications and elaborations available when
required to help prompt tutors. The latter could be of equal interest to
students in directing their efforts to get useful feedback.
This document would be advisory: official department policy is represented by
the
course documentation ("handbook") and the class tutors.
This document is intended to be helpful, but in the end it represents only my
understanding, although it has been widely discussed. Comments, suggested
revisions, and examples, are welcomed from both staff and students.
(To be circulated on/with the marks sheets with all marking)
Students use feedback to improve their performance at writing essays, exam
questions, critical reviews, and project reports. Feedback is helpful for
this formative purpose to the extent that students can see what they could do
differently. It is thus most helpful to focus attention on specific parts or
issues (be diagnostic), and where possible to show explicitly what might have
been done instead (be constructive).
Written feedback (e.g. email or printout) may be more useful than verbal, as
students may consult it later when next doing an exercise of the same kind.
It also lets the tutor send the same comment, where applicable, to more than
one student without duplicating effort.
Do not neglect:
- Positivity: Say what was good as well as what was bad (both for
motivational reasons, so that students can build on their strengths, and to
focus remedial actions on other parts).
- Diagnosticity: Comment both on the overall work, and on particular
sections or aspects separately (e.g. the introduction was satisfactory, the
conclusion very good, but the results should have included ...).
- Constructiveness: Where something was not good, indicate specifically if
at all possible what might or should have been done (e.g. "there was too
little critical comment as opposed to summary. You might, for instance, have
contrasted ... ").
Further material on formative feedback to students: is available at
http://www.psy.gla.ac.uk/~steve/resources/ffeed.html#notes
This document is about what makes useful formative feedback. It is intended
to help tutors produce good quality feedback for students. Equally, however,
it can help students to monitor and manage getting useful feedback for
themselves: for instance by suggesting specific questions they could ask of
their tutors, or of other students (asking for and focussing peer feedback),
and of themselves -- often we can produce most of the feedback we need
ourselves. Indeed, a crucial part of learning any subject or skill is
internalising an ability to judge our own (and others') performance at
it. However others' feedback is an important supplement, particularly in
improving our judgement.
Summative feedback is about comparing finished pieces of work and
judging the resulting quality. Consumer magazines produce summative judgements
to help their readers choose between products (not to improve the products);
and universities assess students to provide accreditation: publicly vouching
for the standard attained by that student. Formative feedback, in contrast,
aims to help
improve the work being evaluated: whether to help improve the design of a
product, help a student improve their essay writing, or help a lecturer improve
their teaching. For more on formative vs. summative and other distinctions,
see these notes
with pointers to the literature. (The discussion there is about evaluation
of computer assisted learning, but the distinctions apply to evaluation of
anything, including a student essay.)
This document is about formative feedback to students: feedback whose whole
purpose is to help the student improve their future work. This is in
contrast to giving marks towards a degree, which is part of summative
evaluation and the assessment and accreditation functions of a university,
and where the marks are often witheld from the student as they are not
finalised until approved by the examiners' meeting held with the external
examiner. It is also in contrast to giving a mark that is passed to the
student, which again is summative evaluation: this tells the student
something about how well they did a piece of work without giving any
information about what to do differently. Since there is almost always
something that could be done better, and as learning to do better regardless
of whether the initial performance was good or bad is presumably part of
getting the most out of a course, providing formative feedback regardless of
whether marks are also provided is an important function.
Students spend two full years in levels 3 and 4, and while they receive
significant amounts of formative feedback from tutors while (rather than
after)
working on critical reviews and projects, the amount of formative feedback
they get on completed work is very limited. Currently (December 1999) it is
only on one critical review and one (2?) miniproject. Even if we were to
overcome university resistance and give formative feedback on three critical
reviews and two miniprojects, this would be at most five pieces of formative
feedback. This is all the formative feedback given on essay writing (the
only mode of exam in honours) and on the assessed coursework (critical
reviews and maxiproject). This seems a small amount to guide the entire two
years' work in honours. Since these exercises are significantly different
from the work required in levels 1 and 2, and no pre-requisite of one hour or
4,000 word essay writing is tested and insisted on for entry to honours,
these exercises and formative feedback on them seem vital for the honours
programme.
More and more, I find this a good heuristic to use in all feedback
situations. It is a good discipline for the feedback giver: it prompts them
to consider explicitly both sides of the issue, and so to come to a more
balanced overall assessment. It helps overcome both a bias to being negative
(that can come from thinking that feedback means critiquing, and critiquing
just means listing what's bad), and a bias to being polite. Furthermore it
is genuinely informative for the receivers: they are sometimes quite
as unaware of their good points, or of just how good the best bit of their
work is, as they are of their bad points.
Thus one of the benefits of this is to allow the recipients to build on their
strengths, and not to lose them in addressing their weaknesses. Another, of
course, is motivational. It is depressing to hear only what you did wrong;
but also this is usually an unbalanced view, and if taken too literally as
meaning you did nothing right, would be quite misleading and could lead to
abandoning satisfactory practices along with unsatisfactory ones. Thus,
thirdly, a benefit of positive along with negative feedback is to contribute
to diagnosticity: to focussing attention on (just) the parts most needing
change.
Finally, simply saying "good", rather than stating specifically what is good
about it, can end up suggesting the wrong message. One word of praise and a
paragraph of criticism does not look as if the tutor does think it's good.
One word of praise and nothing else at all can look as if the tutor doesn't
think it worth thinking about really, as if it means "good for a 2.2, no hope
of trying for anything more": in other words, not very good at all.
To be formative, that is to help students see what and how they could have
done better and so to improve their practice at this work in future, feedback
needs to be diagnostic. That is, it needs to point as specifically as
possible to the part or aspect or thing to be improved.
(There are really two parts to being diagnostic, both separately helpful:
saying what (where) the bad thing is, and saying what is bad about it (why).)
So important is this, that programming language compilers achieve most of their
usefulness like this: their statements about what is wrong with a piece of
code are often pretty inaccurate from the programmer's viewpoint, but nearly
always they point to the exact line in the code where the problem is, and given
that highly specific pointer, a programmer can pretty quickly see what needs to
change. Conversely, course feedback to lecturers from the standard course
feedback questionnaires widely used across the world has been shown to be
largely ineffective in leading to improvements in teaching (Marsh, 1987, p.338;
Marsh et al. 1975).
This is probably because these questionnaires are low on diagnosticity.
If I'm told, after a block of 20 lectures, that many students see me as low on
a "well organised" dimension, this doesn't fill me with ideas on what to
change. A different kind of feedback that told me that lectures 12 and 13 were
bad because no-one knew what relationship they had to the rest of the course,
and that these stood out because the others had been adequate in this respect,
would make it much clearer what to focus my attempts to change on.
Similarly, telling a student just "good" or "uninspiring", gives little clue
about what and how to improve. A good start at improving on this is to narrow
down each piece of praise and criticism to a section or issue (i.e. a position
in the text, or a separable aspect of it).
A simple prompt list might thus be:
- introduction,
- main argument,
- conclusion.
A longer one might add on:
- title,
- abstract,
- references,
- use and choice of section titles.
A list for a project report might include:
- introduction (including literature background),
- method (including
- experimental design,
- participants,
- procedure,
- method of assigning to groups,
- the experimental tasks, etc.),
- results (including appropriate and sufficient use of graphs and tables),
- discussion.
A more
abstract checklist, perhaps as a separate cross check, might include:
- understanding of the concepts and issues, presentation of important details,
- original ideas (or other evidence of critical thought and deep thinking),
- coherence of the argument,
- spelling, grammar and sentence clarity,
- printout format (margins, font size and spacing, page numbers etc.etc.).
However considering commenting on each part and/or aspect doesn't rule out
commenting on the whole as well, not least because it is frequently the
omission of something not on any standard checklist that is the most important
comment to make (e.g. "despite the section title, you do not seem to have a
real conclusion to end with: no sense of closure or where we are now", or "you
do not mention X which you could have discussed either in the introduction, or
in the conclusion as a contrast to your own view").
As useful as narrowing down what to change is to communicate
how it could be changed (for the better): "being constructive".
It is sufficient that the student has a way to know this. Thus I seldom specify
spelling errors, though I often indicate whether they have none, one per
page, or one per line, but assume that dictionaries and spell checkers will
suffice except when they probably won't ("principal and principle are
different", "you can't use apostrophes correctly: look them up in Fowler's
Modern English usage"). Equally, a report completely failing to
follow a standard structure can be referred to a book on how to write
experimental reports, and criticisms of needing more critical thought could
be referred to the
Critical Review document on the web for example directions.
Other issues may well need explicit suggestions on how to improve.
As discussed above, there are probably three major factors contributing to the
effectiveness of formative feedback (i.e. to the likelihood of the recipient
improving their performance): locating the bad thing, saying what is wrong
with it, and saying how it could be fixed. Feedback that gives none of these
(i.e. just signals that something is wrong somewhere in there),
seems to be largely ineffective (e.g. giving a student essay a mark and
nothing else, basic course feedback to lecturers). Doing even one of these
three well is often enough to make a significant difference e.g. compiler
error messages (good only at locations). Papers looking at the effectiveness
of feedback to lecturers claim to see benefits, but they have to admit
improvements are small ("...impact of the feedback was positive, but very
modest", Marsh et al., 1975). However adding consultations about how to improve
things along with the questionnaire feedback (which I see as being poor in
diagnosticity) shows large improvements (Overall & Marsh, 1979).
It seems, then, that big gains can be had by getting beyond a general signal
by adding at least one of these three factors. In principle, addressing all
three seems desirable, but in many cases it is clearly unnecessary.
For instance in most (but not all) cases, pointing out that a particular word
is wrongly spelled without saying what the right spelling is, is sufficient.
Conversely, in many (but not all) cases, suggesting a writing tactic (e.g.
relating their review to the nearest published review) without saying where
this should be inserted, will work.
When I mark an exam script, I imagine having to justify the mark to an
examiners' meeting or a second marker. In particular, I ask myself: what
evidence is there that it should get more than 0%? what evidence that it
should get less than 100%? This kind of approach is also helpful in
generating, not marks, but formative feedback because it prompts me to list
positive achievements in the first case, and examples of exactly what could
have been done better in the second. If I am not going to award a mark
equivalent to 100% I should be able to state in detail what could have been
done to earn 100%, and that should be possible for a student i.e. not require
more time and effort than is supposed to be allocated for the exercise, nor
reading 100 papers on the topic, nor access to knowledge not in the university
library. I often find this a challenging standard, especially for pieces of
work that are adequate but uninspiring; but it forces me to give more helpful
feedback.
In practice, tutors have limited energy for commenting on a single piece of
work, and in any case the student almost certainly has a limited capacity to
deal with it. It is thus obviously sensible to impose an order of importance
on the issues that arise, and focus only on the most important. (E.g. no need
or time to quibble about the reference style when the piece completely fails to
introduce the topic to the reader.)
A consequence of this is that the typical shape of formative feedback for a
generally poor and a generally excellent piece of work can be rather
different. Excellent work often leads to comments with a shape like "Very
good in all respects, which leaves me with little to say. I suppose one
thing you could have added, is to have discussed explicitly the nearest
published review and the way in which your review differed from it ....". On
the other hand a generally frightful bit of work is a challenge of priorities
e.g. "The very large number of spelling errors gives the impression that you
didn't care to do well in this: why not use a spelling checker? The lack of
any section headings and worse, of much sign of any real structure to the
writing, is more serious: either consult a book on how to write essays or at
least adopt the suggestions in the CR document on the web. Your commentary
on each individual paper was superficial: if you can or wish only to deal
with each in this way, then you need to cover a large number of papers (at
least ten). Alternatively, concentrate on a few and have a much deeper
analysis of each. Your introduction made me wonder if you understood the
main issue in this area: surely you at least read .... Those are by no
means all the problems, but they are the most important to deal with
first.".
Burchfield,R.W. (ed.) (1996) The new Fowler's modern English usage
3rd edition (Oxford: Clarendon Press)
Draper,S.W. (1999) Critical Reviews [WWW document]. URL
http://www.psy.gla.ac.uk/~steve/resources/crs.html
Marsh,H.W. (1987) "Students' evaluations of university teaching: Research
findings, methodological issues, and directions for future research"
International journal of educational research
vol.11 no.3 pp.253-388
Marsh,H.W., Fleiner,H. & Thomas,C.S. (1975) "Validity and usefulness of
student evaluations of instructional quality"
Journal of educational psychology vol.67 pp.833-839
Overall,J.U. & Marsh,H.W. (1979) "Midterm feedback from students:
Its relationship to instructional improvement and students' cognitive
and affective outcomes"
Journal of educational psychology vol.71 pp.856-865
Web site logical path:
[www.psy.gla.ac.uk]
[~steve]
[resources]
[this page]
[Top of this page]