Methods & Meta-science

Mixed-effects modeling of temporal autocorrelation: Keep it minimal

Linear models assume the independence of residuals. This assumption is violated in datasets where there is temporal autocorrelation among the residuals. Recently, Baayen, Vasishth, Kliegl, and Bates (2017) presented evidence for autocorrelation in three psycholinguistic datasets, and showed how Generalized Additive Mixed Models (GAMMs) can be used to model these autocorrelation patterns. However, there is currently little understanding of the impact of autocorrelation on model performance, and the extent to which GAMMs improve (or impair) inference. Through Monte Carlo simulation, we found that mixed-effects models perform well in the face of autocorrelation, except when autocorrelation is confounded with treatment variance. GAMMs did little to improve power, and in fact, the use of factor smooths dramatically impaired power for detecting effects of any between-subjects factors. These results suggest GAMMs are only needed in special cases.