Supporting joint human-computer judgment under uncertainty

In this paper we present a concept and interface design aimed at combining expert human judgment with computational support. The goal of this design is to leverage the strengths and simultaneously compensate for the weaknesses of both the expert and a computational model. In order to test the design, we created a task modeled after fantasy baseball, which requires competitors to predict the performance of actual Major League Baseball (MLB) players over the course of the season. The most substantial and challenging aspects of the design involved how to both welcome expert input on a case-by-case basis, yet also provide visual guidance for how these inputs should reflect an appropriate degree of regression to the mean, or reliance on base-rate information. Results showed that the joint human-model system resulted in better performance than a model, which was based, in part, on past performance. The joint system also outperformed unaided or partially-aided experts in some cases but only equally as well in other cases. Design implications and future directions are discussed.