TY - JOUR

T1 - TUTORIAL

T2 - “With sufficient increases in X, more people will engage in the target behavior”

AU - Regenwetter, Michel

N1 - Funding Information:
This work was funded by National Science Foundation (NSF), United States grant SES-14-59699 (PI: M. Regenwetter). The QTest software was additionally funded by NSF, United States grant SES 10-62045 (PI: M. Regenwetter), as well as by the Humboldt Foundation, Germany (Co-PIs: J. Stevens and M. Regenwetter). Final Bayes factor computation, convergence analysis, and a revision of the manuscript were supported by the Army Research Office (ARO), United States, under Multidisciplinary University Research Initiative (MURI) Grant Number W911NF-20-1-0252 (PI: C. Langbort, Co-PIs: M. Başar, M. Regenwetter). NSF, Humboldt and ARO played no other role besides financial support.

PY - 2020/12

Y1 - 2020/12

N2 - Psychological theory should guide the method. A method should not dictate theory. Extraneous assumptions entering psychological theories through the backdoor of a method may differentially affect the analysis of different data sets. This introduces noise and jeopardizes successful replication of valid theoretical claims. Auxiliary theoretical assumptions can also bias substantive conclusions (including across replications). It is therefore becoming ever more crucial that theoretical claims genuinely represent the given theory, no more, no less. Recent work has highlighted a disconnect between some theories and their ‘predictions,’ questioned the scope of theories in the presence of heterogeneity in hypothetical constructs, and developed methods to avoid extraneous assumptions. This tutorial merges these strands of research using a simple, illustrated case study on formulating and testing order-constrained theories. The tutorial applies to empirical paradigms in which scholars can state ordinal constraints on the outcome probabilities for several binary variables such as binary responses or the presence/absence of symptoms, and where the collection of binary variables is associated with a finite set of distinct conditions, such as group membership, treatment condition, or discrete levels of an independent variable. The goal is to let scholars spell out very precise hypotheses that (1) areunadulterated reflections of their theory, (2) provide exceptional theoretical nuance, (3) formally accommodate substantive heterogeneity and (4) offer rigorous and strong quantitative diagnosticity.

AB - Psychological theory should guide the method. A method should not dictate theory. Extraneous assumptions entering psychological theories through the backdoor of a method may differentially affect the analysis of different data sets. This introduces noise and jeopardizes successful replication of valid theoretical claims. Auxiliary theoretical assumptions can also bias substantive conclusions (including across replications). It is therefore becoming ever more crucial that theoretical claims genuinely represent the given theory, no more, no less. Recent work has highlighted a disconnect between some theories and their ‘predictions,’ questioned the scope of theories in the presence of heterogeneity in hypothetical constructs, and developed methods to avoid extraneous assumptions. This tutorial merges these strands of research using a simple, illustrated case study on formulating and testing order-constrained theories. The tutorial applies to empirical paradigms in which scholars can state ordinal constraints on the outcome probabilities for several binary variables such as binary responses or the presence/absence of symptoms, and where the collection of binary variables is associated with a finite set of distinct conditions, such as group membership, treatment condition, or discrete levels of an independent variable. The goal is to let scholars spell out very precise hypotheses that (1) areunadulterated reflections of their theory, (2) provide exceptional theoretical nuance, (3) formally accommodate substantive heterogeneity and (4) offer rigorous and strong quantitative diagnosticity.

KW - Order-constrained hypotheses

KW - Parsimony

KW - Scientific reasoning

KW - Theoretical nuance

KW - Theory formulation

UR - http://www.scopus.com/inward/record.url?scp=85095447727&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85095447727&partnerID=8YFLogxK

U2 - 10.1016/j.jmp.2020.102457

DO - 10.1016/j.jmp.2020.102457

M3 - Review article

AN - SCOPUS:85095447727

VL - 99

JO - Journal of Mathematical Psychology

JF - Journal of Mathematical Psychology

SN - 0022-2496

M1 - 102457

ER -