Learning when to trust which experts

David Helmbold, Stephen Kwek, Leonard B Pitt

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The standard model for prediction using a pool of experts has an underlying assumption that one of the experts performs well. In this paper, we show that this assumption does not take advantage of situations where both the outcome and the experts' predictions are based on some input which the learner gets to observe too. In particular, we exhibit a situation where each individual expert performs badly but collectively they perform well, and show that the traditional weighted majority techniques perform poorly. To capture this notion of ‘the whole is often greater than the sum of its parts’, we propose an approach to measure the overall competency of a pool of experts with respect to a competency class or structure. A competency class or structure is a set of decompositions of the instance space where each expert is associated with a ‘competency region’ in which we assume he is competent. Our goal is to perform close to the performance of a predictor who knows the best decomposition in the competency class or structure where each expert performs reasonably well in its competency region. We present both positive and negative results in our model.

Original languageEnglish (US)
Title of host publicationComputational Learning Theory - 3rd European Conference, EuroCOLT 1997, Proceedings
EditorsShai Ben-David
PublisherSpringer-Verlag
Pages134-149
Number of pages16
ISBN (Print)3540626859, 9783540626855
StatePublished - Jan 1 1997
Event3rd European Conference on Computational Learning Theory, EuroCOLT 1997 - Jerusalem, Israel
Duration: Mar 17 1997Mar 19 1997

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume1208
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other3rd European Conference on Computational Learning Theory, EuroCOLT 1997
CountryIsrael
CityJerusalem
Period3/17/973/19/97

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Learning when to trust which experts'. Together they form a unique fingerprint.

  • Cite this

    Helmbold, D., Kwek, S., & Pitt, L. B. (1997). Learning when to trust which experts. In S. Ben-David (Ed.), Computational Learning Theory - 3rd European Conference, EuroCOLT 1997, Proceedings (pp. 134-149). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 1208). Springer-Verlag.