Randomized incomplete u-statistics in high dimensions

Xiaohui Chen, Kengo Kato

Research output: Contribution to journalArticlepeer-review


This paper studies inference for the mean vector of a high-dimensional U-statistic. In the era of big data, the dimension d of the U-statistic and the sample size n of the observations tend to be both large, and the computation of the U-statistic is prohibitively demanding. Data-dependent inferential procedures such as the empirical bootstrap for U-statistics is even more computationally expensive. To overcome such a computational bottleneck, incomplete U-statistics obtained by sampling fewer terms of the U-statistic are attractive alternatives. In this paper, we introduce randomized incomplete U-statistics with sparse weights whose computational cost can be made independent of the order of the U-statistic.We derive nonasymptotic Gaussian approximation error bounds for the randomized incomplete U-statistics in high dimensions, namely in cases where the dimension d is possibly much larger than the sample size n, for both nondegenerate and degenerate kernels. In addition, we propose generic bootstrap methods for the incomplete U-statistics that are computationally much less demanding than existing bootstrap methods, and establish finite sample validity of the proposed bootstrap methods. Our methods are illustrated on the application to nonparametric testing for the pairwise independence of a high-dimensional random vector under weaker assumptions than those appearing in the literature.

Original languageEnglish (US)
Pages (from-to)3127-3156
Number of pages30
JournalAnnals of Statistics
Issue number6
StatePublished - 2019


  • Bernoulli sampling
  • Bootstrap
  • Divide and conquer
  • Gaussian approximation
  • Incomplete U-statistics
  • Randomized inference
  • Sampling with replacement

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty


Dive into the research topics of 'Randomized incomplete u-statistics in high dimensions'. Together they form a unique fingerprint.

Cite this