Choice of the Berger and Boos confidence coefficient in an unconditional test for equality of two binomial probabilities

Exact unconditional tests for comparing two binomial probabilities are generally more powerful than conditional tests like Fisher’s exact test. Such tests can be further improved by the Berger and Boos confidence interval method, where a p-value is found by restricting the common binomial probability under H0 to a 1-gamma confidence interval. Different default values of gamma, such as 10-3, 10-4, and 10-6, have been used in software. We studied average test power for the exact unconditional z-pooled test for a wide range of cases with balanced and unbalanced sample sizes, and significance levels 0.05 and 0.01. Among the values 10-3, 10-4, …, 10-10, the value gamma = 10-4 was optimal or approximately optimal in all the cases we looked at, and can be given as a general recommendation. Reference Lydersen, S., Langaas, M., & Bakke, Ø. The Exact Unconditional z-pooled Test for Equality of Two Binomial Probabilities: Optimal Choice of the Berger and Boos Confidence Coefficient. Journal of Statistical Computation and Simulation. Available online: 04 Jul 2011.