An exhaustive numerical assessment of alternative unconditional tests of a binary treatment effect

Chris J. Lloyd, Enrico Ripamonti

Research output: Contribution to journalArticlepeer-review

Abstract

For testing the effectiveness of a treatment on a binary outcome, a bewildering range of methods have been proposed. How similar are all these tests? What are their theoretical strengths and weaknesses? Which are to be recommended and what is a coherent basis for deciding? In this paper, we take seven standard but imperfect tests and apply three different methods of adjustment to ensure size control: maximization (M), restricted maximization (B) and bootstrap/estimation (E). Across a wide conditions, we compute exact size and power of the 7 basic and 21 adjusted tests. We devise two new measures of size bias and intrinsic power, and employ novel graphical tools to summarise a huge set of results. Amongst the 7 basic tests, Liebermeister’s test best controls size but can still be conservative. Amongst the adjusted tests, E-tests clearly have the best power and results are very stable across different conditions.

Original languageEnglish
Pages (from-to)1-20
Number of pages20
JournalJournal of Statistical Computation and Simulation
DOIs
Publication statusE-pub ahead of print - Mar 22 2018

Keywords

  • Berger–Boos
  • bootstrap
  • exact tests
  • intrinsic power
  • Liebermeister test
  • M test
  • mid-p
  • size bias

ASJC Scopus subject areas

  • Statistics and Probability
  • Modelling and Simulation
  • Statistics, Probability and Uncertainty
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'An exhaustive numerical assessment of alternative unconditional tests of a binary treatment effect'. Together they form a unique fingerprint.

Cite this