Criteria for performance evaluation

J. Weiss David, Kristin Brennan, Thomas Rick, Kirlik Alex, Sarah M. Miller

Research output: Contribution to journalArticlepeer-review


Using a cognitive task (mental calculation) and a perceptual-motor task (stylized golf putting), we examined differential proficiency using the CWS index and several other quantitative measures of performance. The CWS index (Weiss & Shanteau, 2003) is a coherence criterion that looks only at internal properties of the data without incorporating an external standard. in Experiment 1, college students (n = 20) carried out 2- and 3-digit addition and multiplication problems under time pressure. in Experiment 2, experienced golfers (n = 12), also college students, putted toward a target from nine different locations. Within each experiment, we analyzed the same responses using different methods. For the arithmetic tasks, accuracy information (mean absolute deviation from the correct answer, MAD) using a coherence criterion was available; for golf, accuracy information using a correspondence criterion (mean deviation from the target, also MAD) was available. We ranked the performances of the participants according to each measure, then compared the orders using Spearman's rs. For mental calculation, the CWS order correlated moderately (rs =.46) with that of MAD. However, a different coherence criterion, degree of model fit, did not correlate with either CWS or accuracy. For putting, the ranking generated by CWS correlated .68 with that generated by MAD. Consensual answers were also available for both experiments, and the rankings they generated correlated highly with those of MAD. The coherence vs. correspondence distinction did not map well onto criteria for performance evaluation.

Original languageEnglish (US)
Pages (from-to)164-174
Number of pages11
JournalJudgment and Decision Making
Issue number2
StatePublished - Mar 2009


  • Arithmetic
  • Cws index
  • Judgment
  • Measurement

ASJC Scopus subject areas

  • General Decision Sciences
  • Applied Psychology
  • Economics and Econometrics


Dive into the research topics of 'Criteria for performance evaluation'. Together they form a unique fingerprint.

Cite this