Not so fast! Premature conclusions in cognitive neuroscience and beyond
In this talk I discuss four common statistical errors that are particularly relevant for cognitive neuroscience. The first error concerns the fact that the difference between “significant” and “not significant” is itself not necessarily significant. The second error concerns the overinterpretation of 2 x 2 interactions that do not cross. The third error concerns the misinterpretation of the p-value as evidence against the null hypothesis; specifically, I will show that when p is about .05, the real evidence against the null is anecdotal at best. The fourth error is perhaps most serious, and it concerns the presentation of exploratory studies as confirmatory. The third and fourth error suggest that cognitive neuroscientists should change the way they carry out their experiments and analyze their data.