Characterizing the function space for bayesian kernel models

Natesh S. Pillai, Qiang Wu, Feng Liang, Sayan Mukherjee, Robert L. Wolpert

Research output: Contribution to journalArticlepeer-review


Kernel methods have been very popular in the machine learning literature in the last ten years, mainly in the context of Tikhonov regularization algorithms. In this paper we study a coherent Bayesian kernel model based on an integral operator defined as the convolution of a kernel with a signed measure. Priors on the random signed measures correspond to prior distributions on the functions mapped by the integral operator. We study several classes of signed measures and their image mapped by the integral operator. In particular, we identify a general class of measures whose image is dense in the reproducing kernel Hilbert space (RKHS) induced by the kernel. A consequence of this result is a function theoretic foundation for using non-parametric prior specifications in Bayesian modeling, such as Gaussian process and Dirichlet process prior distributions. We discuss the construction of priors on spaces of signed measures using Gaussian and Levy processes, with the Dirichlet processes being a special case the latter. Computational issues involved with sampling from the posterior distribution are outlined for a univariate regression and a high dimensional classification problem.

Original languageEnglish (US)
Pages (from-to)1769-1797
Number of pages29
JournalJournal of Machine Learning Research
StatePublished - Aug 2007


  • Dirichlet processes
  • Gaussian processes
  • Integral operator
  • Lévy processes
  • Non-parametric Bayesian methods
  • Reproducing kernel Hilbert space

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Statistics and Probability
  • Artificial Intelligence


Dive into the research topics of 'Characterizing the function space for bayesian kernel models'. Together they form a unique fingerprint.

Cite this