Learning when to trust which experts

David Helmbold, Stephen Kwek, Leonard Pitt

Research output: Chapter in Book/Report/Conference proceedingConference contribution


The standard model for prediction using a pool of experts has an underlying assumption that one of the experts performs well. In this paper, we show that this assumption does not take advantage of situations where both the outcome and the experts' predictions are based on some input which the learner gets to observe too. In particular, we exhibit a situation where each individual expert performs badly but collectively they perform well, and show that the traditional weighted majority techniques perform poorly. To capture this notion of ‘the whole is often greater than the sum of its parts’, we propose an approach to measure the overall competency of a pool of experts with respect to a competency class or structure. A competency class or structure is a set of decompositions of the instance space where each expert is associated with a ‘competency region’ in which we assume he is competent. Our goal is to perform close to the performance of a predictor who knows the best decomposition in the competency class or structure where each expert performs reasonably well in its competency region. We present both positive and negative results in our model.

Original languageEnglish (US)
Title of host publicationComputational Learning Theory - 3rd European Conference, EuroCOLT 1997, Proceedings
EditorsShai Ben-David
Number of pages16
ISBN (Print)3540626859, 9783540626855
StatePublished - 1997
Externally publishedYes
Event3rd European Conference on Computational Learning Theory, EuroCOLT 1997 - Jerusalem, Israel
Duration: Mar 17 1997Mar 19 1997

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Other3rd European Conference on Computational Learning Theory, EuroCOLT 1997

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'Learning when to trust which experts'. Together they form a unique fingerprint.

Cite this