Grassroots Training for Reproducible Science
Psychology has received much criticism of late, with classic findings failing to replicate, and an appreciation that many published findings may be unreliable. Small sample sizes and questionable research practices such as p-hacking, hypothesising after the results are known, and selectively publishing positive results have all been implicated as maintaining factors. In response there is a move towards more rigorous and transparent scientific practices and better education is an important means to achieve this. Undergraduate courses in psychology have an assessed research component, where each student undertakes a research project. Given the timescale and resources these project are often small, suffering from many of the well documented problems associated with small sample size (i.e., low power to detect genuine effects, and inflated chance of finding false ones). The sheer number of these projects, coupled with the potential for undisclosed analytic flexibility, means the results from these studies may be unreliable, and if the lucky few who do find ‘significant’ results selectively go on to publish, then they further undermine the reliability of the evidence base. Solutions to address these problems include pre-registration of study protocols and analysis plans, and designing studies with sufficient statistical power. However, such studies often require resources beyond those available for the typical undergraduate student project. One solution to this is collaboration. Here I present the rationale for a consortium-based approach to undergraduate projects, and some insights from the first GW4 Undergraduate Psychology Consortium currently in progress. By working together, students and academics can conduct rigorous research that is of such high quality that it stands a good chance of being published regardless of the results, thus mutually satisfying both the drive to publish and the drive for increased methodological rigour.