Towards Meaningful Inferences From Attitudinal Thermometer Ratings

Thermometer ratings and Likert scales are ubiquitous in social psychology, political psychology, and political science, even though critics have cautioned that researchers take the scores too literally. A measurement procedure based on arbitrary assumptions risks the real danger of generating scientifically meaningless inferences. Adopting a decision theoretic point of view, we use the concept of semiorders to capture the idea that a person giving 2 candidates distinct scores might or might not actually prefer one to the other, depending on the size of her threshold of discrimination. Furthermore, one respondent giving a candidate a lower score than another respondent could nevertheless be the stronger supporter. We state formal assumptions about the nature of preferences and propose a novel probabilistic response mechanism by which respondents construct numerical scores heterogeneously when asked to represent their preferences in a numerical format. We provide a proof of concept using maximum likelihood tests of our models on public domain American National Election Study data.