Multiplicity Control vs Replication
Making an Obvious Choice Even More Obvious
Keywords:multiplicity control, familywise error, power, replication, effect size, meta-analysis
This paper presents a side-by-side consideration of multiplicity control procedures and replication as solutions to the problem of multiplicity. Several independent theoretical arguments are presented which demonstrate that replication serves several important functions, and that multiplicity control procedures have a number of serious flaws. Subsequently, the results of a simulation study are provided, showing that under typical conditions, replication provides similar familywise error control and power as multiplicity control procedures. Taken together, these theoretical and statistical arguments lead to the conclusion that researchers who are concerned about the problem of multiplicity should shift their attention away from multiplicity control procedures and towards increased use of replication.
Copyright (c) 2020 Andrew Hunter, Nataly Beribisky, Linda Farmus, Robert A. Cribbie
This work is licensed under a Creative Commons Attribution 4.0 International License.