Reply to Commentaries: Why Should We Worry About Scientific Conjunction Fallacies?

We agree with Erev and Feigin (2022) that one should model heterogeneity at different levels. We do not promote either distribution-first or individual-first approaches over the others because population-level heterogeneity compounds sources of heterogeneity. We qualify Erev and Feigin’s proposals in that neither approach is immune to scientific reasoning errors. We agree with Scheibehenne (2022) that aggregate statistics can appropriately summarize behavior, provided that the “effects” are robust across individuals. In contrast to this idealized scenario, decision researchers often deal with the joint occurrence of important qualitative differences on numerous attributes. Misconstruing individual differences as error variance carries a cost and violates the definition of overfitting. We agree with Kellen (2022) that the literature is often vague enough not to state verbatum that CPTMED is more descriptive of behavior than CPT with free parameters, but scientific conjunction errors are not so limited in scope. Regenwetter et al. (2022) intentionally glossed over potential limitations of studies, such as response errors, sample quality, reliability of measures, and diagnosticity of stimuli, to make a conceptual point. Speculating about the joint influence of these factors would render us agnostic about the lower and upper bounds on the number of people who satisfy a stylized theory. “Recipes” in study design are not exempt from conjunction errors: Fallacious reasoning is pernicious at any stage of scientific theorizing. We agree with Kellen that efforts to lead decision research beyond stylized theory deserve much further attention and future work.