Evaluation of methods for analysis of 2x2 contingency tables

Evaluation of methods for analysis of 2x2 contingency tables Introduction: Literally hundreds of methods for hypothesis tests and confidence intervals for contingency tables are described in the literature. This is the case even for the seemingly simple 2 × 2 table. The Pearson chi squared test, the Fisher exact test, and the Wald confidence interval are widely used methods. Unfortunately, these methods are also commonly used in situations when they perform poorly, and better alternatives exist. Aims: To show how to evaluate and choose appropriate methods for hypothesis testing and confidence interval estimation for 2x2 tables Methods: Important properties of a hypothesis test are the actual significance level and power. Important properties for a confidence interval are coverage probability, expected interval width, and symmetry. We have studied these properties for alternative methods. Conclusion: For large samples, the asymptotic Pearson chi squared test performs well. Yates’s continuity correction should never be used. In small samples, the traditional advice is the Fisher exact test. But unconditional tests are generally more powerful than Fisher’s exact test for small samples. Unconditional tests also preserve the significance level. That is, the actual significance level does not exceed the nominal significance level, which is often 5%. Unconditional tests were but previously disadvantaged by being computationally demanding. Alternatively, Fisher’s exact test with mid-p adjustment is easy to compute, and gives approximately the same results as an unconditional test. The Wald confidence interval is a traditional choice for the difference between two proportions. But its coverage can dip substantially below the nominal coverage, which is usually 95%. The Agresti-Caffo interval and the Newcombe hybrid score interval are also easy to compute with closed form expressions. They have better coverage, and also narrower expected interval width, and better symmetry than the Wald interval. Reference: Fagerland MW, Lydersen S, Laake P (2017). Statistical Analysis of Contingency Tables. Chapman and Hall/CRC.