Low statistical power (see step 3) | The sample size is not adequate to detect an effect. |

Violated assumptions of statistical tests (see step 6) | The data violate assumptions needed for the test, such as normality. |

Fishing and error rates | Repeated tests of the same data (eg, multiple comparisons) increase chances of errors in conclusions. |

Unreliability of measures | Error in measurement or instruments can artificially inflate or decrease apparent relationships among variables. |

Restricted range | Statistics can be biased by limited outcome values (eg, high/low only) or floor or ceiling effects in which participants scores are clustered around high or low values. |

Unreliability of treatment implementation | In experiments, unstandardised or inconsistent implementation affects conclusions about correlation. |

Extraneous variance in an experiment | The setting of a study can introduce error. |

Heterogeneity of units | As participants differ within conditions, standard deviation can increase and introduce error, making it harder to detect effects. |

Inaccurate effect size estimation | Outliers or incorrect effect size calculations (eg, a continuous measure for a dichotomous dependent variable) can skew measures of effect. |