The construct-behavior gap in behavioral decision research: A challenge beyond replicability

Behavioral decision research compares theoretical constructs like preferences to behavior such as observed choices. Three fairly common links from constructs to behavior are (1) to tally, across participants and decision problems, the number of choices consistent with one predicted pattern of pairwise preferences; (2) to compare what most people choose in each decision problem against a predicted preference pattern; or (3) to enumerate the decision problems in which two experimental conditions generate a 1-sided significant difference in choice frequency 'consistent' with the theory. Although simple, these theoretical links are heuristics. They are subject to well-known reasoning fallacies, most notably the fallacy of sweeping generalization and the fallacy of composition. No amount of replication can alleviate these fallacies. On the contrary, reiterating logically inconsistent theoretical reasoning over and again across studies obfuscates science. As a case in point, we consider pairwise choices among simple lotteries and the hypotheses of overweighting or underweighting of small probabilities, as well as the description-experience gap. We discuss ways to avoid reasoning fallacies in bridging the conceptual gap between hypothetical constructs, such as, for example, "overweighting" to observable pairwise choice data. Although replication is invaluable, successful replication of hard-to-interpret results is not. Behavioral decision research stands to gain much theoretical and empirical clarity by spelling out precise and formally explicit theories of how hypothetical constructs translate into observable behavior.