23 May 2021 ............... Length about 1,000 words (10,000 bytes).
(Document started on 25 Jan 2018.)
This is a WWW document maintained by
Steve Draper, installed at http://www.psy.gla.ac.uk/~steve/educ/meth.html.
You may copy it.
How to refer to it.
Web site logical path:
Other related pages:
[Invite to seminar]
Linked circle of topics relevant to CCSE:
Department of Psychology,
University of Glasgow.
This page overlaps with another /~steve/best/rct.html
and I'm not sure whether to merge them.
- Ideas about different learning trajectories
The different relationships between the intervention and the measured effect.
MBarr: they DO catch up naturally
They don't catch up naturally
Assymmetric learning effects:
why counter-balancing doesn't work on humans
- Learning to learn
Jigsaw; Ann Brown.
A&S; Papert; diSessa; = CogAcc.
How in fact in most disciplines, learning how to learn more is a central outcome.
E.g. compSci students leave knowing how to learn another programming language
Not learning the subject matter, but learning to learn.
A&S is a pure example of "cognitive acceleration": of training pupils on
materials that for 2 years had no demonstrable effect, but after the training,
then they learned STEM material in the year following much faster than
control group kids did. The intellectual equivalent of spinach for Popeye;
morning coffee on you.
[See (b, c) of the five main papers, and Adey (2004).]
- Multi-stage research
Much research, especially applied research, goes through multiple stages; and
criticisms that are important at one stage may be largely irrelevant at
another. The design of an experiment needs to be appropriate to the stage of
the overall research programme, not dictated by a single ideal standard for
one-off experiments that are imagined to stand by themselves.
MRC stage model for health interventions
The MRC offers this model on p.4 of:
MRC (2000) A Framework for development and evaluation of RCTs for complex interventions to improve health
- Theory - Pre-clinical.
Explore relevant theory to ensure best choice of interention and hypothesis
and to predict major confounders and strategic design issues.
- Modelling - Phase 1.
Identify the components of the intervention, and the underlying mechanisms by
which they will influence outcomes to provide evidence that you can predict how
they relate to and interact with each other.
- Exploratory trial - Phase 2.
Describe the constant and variable components of a replicable intervention AND
a feasible protocol for comparing the intervention to an appropriate
- Definitive RCT - Phase 3.
Compare a fully-defined intervention to an appropriate alternative using a
protocol that is theoretically defensible, reproducible and adequately
controlled, in a study with appropriate statistical power.
- Long-term implementation - Phase 4.
Determine whether others can reliably replicate your intervention and results
in uncontrolled settings over the long term.
The MRC model is for medical research, but education is likelwise an applied
field partly to do with both theory and controlled experiments: and so it is
interesting to look for comparisons and transfer of methods.
The MRC "phase 3" stage uses RCTs (Randomised Controlled Trials). In
another web page
I comment on RCTs and on issues and variations in study design related to them.
Shayer model for educational research
- Primary effect study: what effects, and how large, can be achieved
using the intervention?
- Replication study: Can the effect be transferred from the
researchers to any other teacher(s)? (Or is it only achievable by the
originator and therefore may depend or be wholly due to some unconsicous skill
on their part?). The test at this point is probably still using a highly
self-selected and unusually enthusiastic and able teacher; but at least it has
- Generalizability study: Can a teacher training course be created for
transferring the intervention, and is it successful in achieving the effects
for pupil attainment even with perhaps reluctant teachers?
This is obviously essential for the research to have any significant effect
Shayer gives this model for applied educational research
on pp.112-3 in Shayer(1992).
Shayer,M. (1992) "Problems and issues in intervention studies" in
Demetriou,A., Shayer,M. & Efklides,A. (eds.)
Neo-Piagetian theories of cognitive development: implications and
applications for education ch. 6 pp.107-121 (London : Routledge)
Less generally, one might see Adey & Shayer's own CASE work as having an
extended number of stages.
- Theoretical work.
- Development of materials derived from the theory, to use in the next stage.
- Primary effect study.
- Replication study: transfer to any other teacher(s)?
- Generalizability: create, develop, and test a teacher training course.
- Roll-out in multiple schools
(test it generalises over institutions as well as teachers)
- Follow-up studies on long term effects.
The Louis Pasteur "model" of types of experiment
Rather different is a model, or rather reported practice, of types or phases
or uses for, experiments offered by one of Louis Pasteur's associates.
In a biography, he said that Pasteur's central activity in life was
experimentation, and they could be classified into three types.
- On entering a new field (which Pasteur did quite a number of times)
he would do numerous experiments recorded only in his lab notebooks, by the
end of which he had arrived at his new theories in this field; and remarkably,
seldom had to change them later. Here he was experimenting to learn, not to
communicate to others.
- Doing roughly what most researchers would do today: to develop
experiments that would convince other scientists as long as they weren't
prejudiced by their own theories. These were often published.
- Doing experiments of such monumental precision, that anyone who tried to
object to their conclusions would simply look hopelessly irrational.
Experiments to utterly crush any opposition. Pasteur did some of these: often
as public demonstrations.
- Individual differences in effects
I.e. does a given intervention have the same effect on everyone, or does it
work on some but not on others?
for an elaboration of this small, but rather general and important, point.
The point is: in most educational interventions, you should not only study the
group means, but also the individuals e.g. report what percent of subjects do
and don't show the effect.
Shayer & Adey (1992,3)
showed such differences in effect of their (CASE) interventions.
Despite a large-ish overall group effect size:
a) Some individual kids show a really big effect, some none at all.
b) About twice as many boys as girls show the effect.
- Shayer,M. & Adey,P.S. (1992) "Accelerating the development of formal
thinking in middle and high school students, III: Testing the Permanency of
Effects" J. of research in science teaching vol.29 no.10 pp.1101-1115
- Shayer,M. & Adey,P.S. (1993) "Accelerating the development of formal
thinking in middle and high school students, IV: Three Years after a Two-Year
Intervention" J. of research in science teaching vol.30 no.4 pp.351-366
Web site logical path:
[Top of this page]