A Brief Guide to Evaluate Replications

Authors

  • Etienne Philippe LeBel University of Western Ontario
  • Wolf Vanpaemel
  • Irene Cheung
  • Lorne Campbell

DOI:

https://doi.org/10.15626/MP.2018.843

Keywords:

transparency, reproducibility, direct replication, replicability, evaluating replications;

Abstract

The importance of replication is becoming increasingly appreciated, however, considerably less consensus exists about how to evaluate the design and results of replications. We make concrete recommendations on how to evaluate replications with more nuance than what is typically done currently in the literature. We highlight six study characteristics that are crucial for evaluating replications: replication method similarity, replication differences, investigator independence, method/data transparency, analytic result reproducibility, and auxiliary hypotheses’ plausibility evidence. We also recommend a more nuanced approach to statistically interpret replication results at the individual-study and meta-analytic levels, and propose clearer language to communicate replication results.

Metrics

Metrics Loading ...

Downloads

Published

2019-06-14

Issue

Section

Original articles