Last changed 26 Aug 2014 ............... Length about 3,000 words (25,000 bytes).
(Document started on 4 Feb 2007.) This is a WWW document maintained by Steve Draper, installed at http://www.psy.gla.ac.uk/~steve/localed/tintoactions.html. You may copy it. How to refer to it.

Web site logical path: [www.psy.gla.ac.uk] [~steve] [localed] [dropout] [this page]

Evidence-based Retentioneering:
Interventions that reduce dropout

By Steve Draper,   Department of Psychology,   University of Glasgow.

Contents (click to jump to a section)

  1. Introduction
  2. Pre-selecting students
  3. Engaging the students in understanding
  4. The magic phone call
  5. Assertive outreach
  6. Pre-arrival e-newsletters
  7. Peer mentoring   Roz Phillips' research   Edith Cowan's ultra-successful scheme
  8. First week projects
  9. Targeted remediation for a computer programming course using student facilitators
  10. Positive psychology

Introduction

Theory is entertaining, and talk is cheap; but what matters for practice (besides also being potentially powerful evidence for theories) is proven reductions in dropout rates. Since these seem few and far between, I have collected here those few I've come across. It is a sub-area that can be studied independently.

(There is no a priori reason to expect that we must be able to do anything to change student dropout and retention: hence the importance of direct, proven successes. Strategies based on improving it might turn out to be like reasoning that if you win the lottery jackpot with your first ticket then your money troubles will be over, or that if you invent a perpetual motion machine then you'll never need to worry about fuel bills again: the logic is correct, but there is no evidence that this is a practical, possible plan of action.)

Pre-selecting students

Although the evidence is not quite direct, by far the biggest effects are that the more strongly you pre-select for the students who are most likely to succeed, the lower your dropout rates. This association between selectivity and dropout rate can be inspected and tested (for instance) in the public data HEFCE publishes on English HEIs (Higher Education Institution). It is clear that the most successful HEIs believe this is cause and effect: typically they boast of having the best students (and staff), but not of having the best teaching that can most transform unpromising material.

Institutional dropout rates vary enormously within a given country (about 1% - 38% in the UK), and the most obvious difference between HEIs is their selectivity. The UK Open University (not usually included in these comparisons) has an even bigger dropout rate of 50%, and essentially does not select its student intake at all. We are unlikely to see any other intervention with anything like such a big effect on dropout as pre-selection.

While the most obvious selectivity is by prior academic achievement (e.g. high A-level scores), it could be that important parts of the selectivity (for achieving low dropout) is actually for pre-attunement to HE. If a student's parents both went to university, preferably the same university; if their school assumed they would go and pre-trained them e.g. to take notes, use the library, to write essays exhibiting critical thinking, etc., then this may make that student more likely to succeed. Furthermore there are associations, almost certainly causal, between wealth and family support on the one hand, and retention on the other. More accurately, different families demonstrate different amounts of commitment to keeping a student in education. Previous academic achievement is a measure of this because it measures their demonstrated commitment to date, and so selecting for achievement is also likely to select for continued support, and against students who may have to leave to support their families which is a common cause of dropout.

This kind of advantage may further extend to the built-in dropout-saving mechanisms within HEIs. A leading cause of dropout, particularly at the more successful HEIs, is mismatch between student and course or subject. Many more students "fix" this by changing courses. Although tough definitions would view this as dropout if it takes an extra year, other definitions and most students themselves do not. However the capacity to change courses and then succeed will be greater the more successful a student is at learning, and the more subjects they have taken earlier (e.g. at A-level) and successfully. Again, pre-selecting for the most successful students is likely to mean they are better able to take advantage of changing courses to avoid dropout.

Related to this is that there is a tradeoff for HEIs between dropout and widening participation. An HEI will usually do better at one, if they do worse at the other. This is rather starkly illustrated by the 2006 HESA figures, as reported by THES on 20 July 2007 p.6-7. The six bottom HEIs at participation are all in the top 10 HEIs at retention (low dropout), while their relationship with the top income for research is much weaker: only 2 of these 6 are also in the top 10 for research returns.

"I am not impressed by the Ivy League establishments. Of course they graduate the best: it's all they take, leaving to others the problem of educating the country. They will give you an education the way the banks will give you money, provided you can prove to their satisfaction that you don't need it." ( Peter De Vries)

Engaging the students in understanding

The evidence part is this. Jim Boyle, Mechanical Engineering, Strathclyde redesigned his first year course (for mechanical engineers) and reduced the dropout rate from about 20% to about 3% where it has stayed for some years now. I don't currently have a reference to a published paper with dropout figures in, but you could contact him about this.

What did he do? He redesigned his course to use a method of teaching and learning which Hake called "Interactive Engagement", and Mazur calls "Peer Instruction". Basically a feature is setting brain teazer questions based on key concepts known to be difficult for students, and using them to provoke peer discussion. (Nowadays this is done using EVS: Electronic Voting Systems.) Jim Boyle's methods are described in some detail, with collected evaluation evidence, in:

Boyle, J.T. & Nicol,D.J. (2003) "Using classroom communication systems to support interaction and discussion in large class settings" Association for Learning Technology Journal vol.11 no.3 pp.43-57 pdf copy
Nicol,D.J. & Boyle, J. T. (2003) "Peer Instruction versus Class-wide Discussion in large classes: a comparison of two interaction methods in the wired classroom" Studies in Higher Education vol.28 no.4 pp.457-473 pdf copy

Could they be generalised to other subjects, and how? This is more speculative. EVS have been applied widely in many ways, of which the Mazur method is only one. The Mazur method really derives from Piaget, and indeed Cardinal Newman argued that peer discussion is more important to university education quality than professors or exams are. However would this always improve dropout rates?

My own guesses would be that the first thing is to look at what the bottleneck to achieving learning is in a particular subject: in basic Newtonian mechanics, a long educational literature shows that truly internalising the meaning of the concepts is problematic for many, perhaps most, students. Jim Boyle's redesign was on the basis of this literature i.e. it was addressing what was established as the main issue in learning that particular subject. So the generalisation of that would be to identify what the main bottleneck to success is in the subject under consideration, and address that. In other words, next to selecting students as likely to succeed anyway, supporting their effective learning of their particular subject may be the most effective type of intervention to improve retention that we could make.

My other guess at a generalisation would be that engaging students academically in their subject is of core importance: giving them the (justified) feeling of understanding regardless of how much this impacts on their test scores.

The magic phone call

There is evidence (from controlled experiments) that contacting students personally by phone reduces dropout rates.

The original promising study

There was a controlled experiment at Mississippi (Anderson & Gates, 2002) that showed that a single phone call expressing concern after a student missed 2 classes in 8 weeks changed (for this group of imperfect attenders) the % getting grade C or better from 55% to 87%. They called this "freshman absence-based intervention". Other research suggests it is the first not second missed class that is the important one. This is a big effect for a single phone call: surely worth the cost. (They employed postgraduates, and trained them at this. It requires the HEI (Higher Education Institution) to obtain attendance records at all classes.) [References: Anderson,C. & Gates,C. (2002) "Freshmen Absence-Based Intervention at The University of Mississippi" Original web page, now gone: http://www.brevard.edu/fyc/listserv/remarks/andersonandgates.htm Local copy of the original   Another article]

Tone and content: e.g. "I understand you missed the last tutorial and we're concerned: I wonder if there is anything we could do to help?" (as opposed to "What is your excuse for missing it?"). Or actually: "One of your instructors is concerned about your absences from his class at such an early time in the semester. Understand that instructors have different criteria for what they deem excessive, which might be far different from your own ideas. I tell you this only to notify you that your instructors do in fact notice when you attend or fail to attend their classes. If you are in need of any type of assistance, whether it be academic or personal, please feel free to contact an advisor in the Academic Support Center located at 22 Road Street (123 4567) or the staff at the University Counseling Center at the V.B. Harrison Building (123 4567). Information regarding Academic Probation, Dismissal and Suspension can be found at in the Undergraduate Catalogue."

In the first trial, about 40% of the class qualified by their absences for the intervention. Of these 87% of those getting the intervention got grade C or better, while only 55% of the control group did. In the second trial (with no control group) only 58% got a C or better: as if there were no effect. In the third trial, 70% got C or better. However that was in the second semester, when perhaps most of the dropouts had already occurred and the remaining students were much less likely to fail. Differences between the definite success of the first trial, and the possible failure of the others include:

My unsuccessful attempt to replicate it

An attempt was made to replicate the Mississippi study at the University of Glasgow in 2003-4 for students taking the first semester, first year psychology course (a class of about 600 students altogether). (Jennifer B. Hume (2004) "Can a single telephone call prevent a student from dropping out?".) The trigger criterion was missing two consecutive tutorials. Those meeting the trigger were then randomly assigned to receive either an email or else a phone call. No differences were found in dropout or exam grades at the end of the semester between the groups. Dropout was defined as either withdrawal, or credit refused for failing to complete work e.g. non-attendance at the exam.

There were big differences in the effectiveness of the two methods in reaching the students and gaining an acknowledgement. There was no sign (in an extensive survey) of any problems in the phone calls being seen as intrusive or unwelcome.

Email group Telephone group
Total no. of students meeting the "trigger" 30 30
No. of students who were definitely reached 5 20
Of these, no. who had already dropped out 2 5
Total no. of dropouts by end of semester 17 17

It seems to me there are 3 conclusions we may draw from this.

The successful Open University study

The Open University ran a big study over 3 years with a control group, involving a total of over 5,000 students, and showed that a single phone call reduced dropout by 4-5%. They had a highly predictive statistical model of who was at risk, but in fact found the same small but statistically significant (p < 0.01) effect for high and low risk student groups alike. The phone call in this study was not triggered by non-attendance, but made before the course started regardless. The content of the phone call was derived from the positive psychology literature, and followed a script about emphasising each student's personal strengths at learning.
Year Total students in trial Increase in retention rates: experimental vs. control group
2002 2866 3.9%
2003 1363 5.1%
2004 987 4.3%
2005 10130 7.6%
Simpson,O. (2004) "The impact on retention of interventions to support distance learning students" Open Learning: The Journal of Open and Distance Learning Volume 19 no.1 2004 Pages 79-95

Simpson,O. (2008) "Motivating learners in open and distance learning: do we need a new theory of learner support?" Open Learning: the journal of open and distance learning vol.23 no.3 pp.159-170

Summary

Thus there is evidence that phone calls, if done carefully, are not resented; and that they can save dropouts. However, even though the original thinking was that these should be targeted at students particularly at risk, in fact the evidence is that they can save dropouts even when untargeted. This is a general point to bear in mind: that targeting often may not in fact be necessary or powerful. The next intervention (e-newsletters) underlines this. Another issue is the content of the phone call. However many pointers suggest that perhaps it is the message of caring implicit in making the call that may be the most "active ingredient" in the intervention, rather than any particular content. (See this extra page for more discussion on implementing interventions along these lines.)

Finally, we should today (2007) consider other media than phones. According to rumour at least, one academic at Hull chases up missing students on Facebook to good effect. Other studies report on using SMS texting.
Dave Harley, Sandra Winn, Sarah Pemberton & Paula Wilcox (2007) "Using texting to support students' transition to university" Innovations in Education and Teaching International Vol.44 No.3 pp.229-241

Assertive outreach

See this.

Pre-arrival e-newsletters

This is an idea from the STAR project.

The idea is to tackle the "no-show" students, who receive an unconditional offer of a place on a course, but never show up; in effect "pre-dropouts". The idea is to send monthly e-newsletters, welcoming the students, telling them a bit about what to do when they arrive, who the key people on the staff are, etc. Web page describing this   Longer report.

The evidence is a year on year change when the e-newsletters were introduced: the no-shows (pre-dropouts) fell from 14 to 3 in the year in question (8.8% to 1.9%).

Peer mentoring

I used to think that there was no direct evidence that peer mentoring reduces dropouts. However the indirect evidence was noteworthy: that peer mentors tend to be the people that students in difficulty go back to at need. If sympathetic contact and support can make a difference to dropout, then it seems likely that peer mentoring schemes have a significant effect. Further details here. That page also points to Roz Phillips' research on this.

However I have now heard of a dramatically successful scheme in the psychology department at Edith Cowan university (in Perth, Western Australia), that has a scheme that has reduced dropout from about 23% to 5%. Further details here.

First week projects

Parmar, D. and Trotter, E. (2005) "Keeping our Students: Identifying factors that influence student withdrawal and strategies to enhance the experience and retention of first year students" Learning and Teaching in the Social Sciences 1(3) pp.149-168.

Trotter, E. and Roberts, C.A. (2006) "Enhancing the Early Student Experience" Higher Education Research and Development Vol.25 No.4 pp.371-386

This evidence is not from a controlled experiment, but from a natural one: analysing two existing courses. Trotter's research idea was to select two courses that were matched i.e. broadly comparable in discipline area, size, etc. except for markedly different retention rates, and then do in depth interviews etc. to discover what factors might be responsible. The most obvious of these, it seems at least to me, was that the low-dropout course used the first week for group projects, while the other offered more traditional induction activities such as a tour of the surrounding city. With hindsight, at least, we can recognise that spending the first week on group projects implicitly covers social integration (getting to know other students well), academic integration (getting to know one staff member well), in the context of an "authentic" task i.e. a task to do both with learning and the course (as opposed to, say, library induction tours, lectures on introductory topics that have low apparent connection with the profession and practice the course is meant to train students for).

Targeted remediation for a computer programming course using student facilitators

This intervention falls short of having a control group, random assignment, and conclusive statistical effects. Nevertheless first results show distinct quantitative indications of effectiveness. (Further results are currently being analysed, and are probably a bit stronger.) In this scheme there is:

Draper, S.W. & Cutts,Q. (2006) " Targeted remediation for a computer programming course using student facilitators" Practice and Evidence of the Scholarship of Teaching and Learning in Higher Education" vol.1 no.2 pp.117-128.

Positive psychology

There have recently been some dramatic studies showing large direct educational effects of positive psychology interventions on educational test scores. It seems very likely that in HE these would translate into lower dropout. These are discussed here.

We have also just achieved such an effect ourselves on the first year programming course. The statistical effect size was about 0.5; and the difference in the means (the increase in exam marks) about half a grade point i.e. substantial.

Web site logical path: [www.psy.gla.ac.uk] [~steve] [localed] [dropout] [this page]
[Top of this page]