Eliciting informative feedback: The peer-prediction method

Nolan Miller, Paul Resnick, Richard Zeckhauser

Research output: Contribution to journalArticlepeer-review

Abstract

Many recommendation and decision processes depend on eliciting evaluations of opportunities, products, and vendors. A scoring system is devised that induces honest reporting of feedback. Each rater merely reports a signal, and the system applies proper scoring rules to the implied posterior beliefs about another rater's report. Honest reporting proves to be a Nash equilibrium. The scoring schemes can be scaled to induce appropriate effort by raters and can be extended to handle sequential interaction and continuous signals. We also address a number of practical implementation issues that arise in settings such as academic reviewing and online recommender and reputation systems.

Original languageEnglish (US)
Pages (from-to)1359-1373
Number of pages15
JournalManagement Science
Volume51
Issue number9
DOIs
StatePublished - Sep 2005
Externally publishedYes

Keywords

  • Electronic markets
  • Honest feedback
  • Proper scoring rules

ASJC Scopus subject areas

  • Strategy and Management
  • Management Science and Operations Research

Fingerprint

Dive into the research topics of 'Eliciting informative feedback: The peer-prediction method'. Together they form a unique fingerprint.

Cite this