Exact and asymptotically robust permutation tests

Eunyi Chung, Joseph P. Romano

Research output: Contribution to journalArticlepeer-review


Given independent samples from P andQ, two-sample permutation tests allow one to construct exact level tests when the null hypothesis is P = Q. On the other hand, when comparing or testing particular parameters θ of P and Q, such as their means or medians, permutation tests need not be level α, or even approximately level α in large samples. Under very weak assumptions for comparing estimators, we provide a general test procedure whereby the asymptotic validity of the permutation test holds while retaining the exact rejection probability α in finite samples when the underlying distributions are identical. The ideas are broadly applicable and special attention is given to the k-sample problem of comparing general parameters, whereby a permutation test is constructed which is exact level α under the hypothesis of identical distributions, but has asymptotic rejection probability α under the more general null hypothesis of equality of parameters. A Monte Carlo simulation study is performed as well. A quite general theory is possible based on a coupling construction, as well as a key contiguity argument for the multinomial and multivariate hypergeometric distributions.

Original languageEnglish (US)
Pages (from-to)484-507
Number of pages24
JournalAnnals of Statistics
Issue number2
StatePublished - Apr 2013
Externally publishedYes


  • Behrens-Fisher problem
  • Coupling
  • Permutation test

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty


Dive into the research topics of 'Exact and asymptotically robust permutation tests'. Together they form a unique fingerprint.

Cite this