What to make of equivalence testing with a post-specified margin?

Authors

  • Harlan Campbell University of British Columbia
  • Paul Gustafson University of British Columbia

DOI:

https://doi.org/10.15626/MP.2020.2506

Keywords:

equivalence testing, non-inferiority testing, confidence intervals, type 1 error, frequentist testing, clinical trials, negative studies, null results

Abstract

In order to determine whether or not an effect is absent based on a statistical test, the recommended frequentist tool is the equivalence test. Typically, it is expected that an appropriate equivalence margin has been specified before any data are observed. Unfortunately, this can be a difficult task. If the margin is too small, then the test's power will be substantially reduced. If the margin is too large, any claims of equivalence will be meaningless. Moreover, it remains unclear how defining the margin afterwards will bias one's results. In this short article, we consider a series of hypothetical scenarios in which the margin is defined post-hoc or is otherwise considered controversial. We also review a number of relevant, potentially problematic actual studies from the clinical trials research, with the aim of motivating a critical discussion as to what is acceptable and desirable in the reporting and interpretation of equivalence tests.

Metrics

Metrics Loading ...

Downloads

Published

2021-09-06

Issue

Section

Original articles