Coherence and correspondence competence: Implications for elicitation and aggregation of probabilistic forecasts of world events

One potentially useful concept that arises in the elicitation and aggregation of probabilistic forecasts is Hammond's (1996) distinction between coherence and correspondence. A study was conducted to test the commonly held assumption that coherence competency, a judge's ability to reason correctly according to the prescriptions demanded by the problem, directly yields correspondence competency, a judge's ability to predict the outcome that actually happens in the external world. The role of a visualization aid in terms of moderating these effects was also examined. Participants who were knowledgeable baseball fans predicted the probability with which their favored team would win the 2011 Major League Baseball World Series, giving a prior probability shortly before the start of the Series, and then sequentially updating their answer as the individual games unfolded over time. Results show that for participants using the visualization, their ability to update probabilities according to the dictates of Bayes' Theorem was correlated with their ability to predict the winner of the 2011 MLB Series - a desirable property that allows for estimation of judges' outcome performance based on more readily available process information.