Impact of questionnaire format on reliability, validity, and hypothesis testing

Madhubalan Viswanathan, Ujwal Kayande, Richard P. Bagozzi, Sam Riethmuller, Shirley Y.Y. Cheung

Research output: Contribution to journalArticlepeer-review

Abstract

Researchers use a variety of questionnaire formats to collect data on measures of constructs for theory testing. For example, researchers may label measures of constructs, present measures of different constructs on different pages, or intersperse items from different constructs. Such questionnaire format choices are often guided by commonly held beliefs and conventions. However, there is little if any empirical research evaluating such different formats, precluding an informed view of the appropriateness of a format for a study. To our knowledge, our research represents the first systematic empirical investigation of how different formats affect research outcomes. We conduct a series of studies to systematically examine the effects of questionnaire format on the psychometric properties of measures of constructs and the relationships between constructs. Using multiple group confirmatory factor analysis (MGCFA), we find that the measures are largely invariant to questionnaire format when using student samples, and recommend that researchers should reconsider the rationale provided for choosing specific formats.

Original languageEnglish (US)
Pages (from-to)465-498
Number of pages34
JournalTPM - Testing, Psychometrics, Methodology in Applied Psychology
Volume24
Issue number4
DOIs
StatePublished - Dec 1 2017

Keywords

  • Measurement
  • Multiple group confirmatory factor analysis
  • Questionnaire formats
  • Reliability
  • Theory-testing

ASJC Scopus subject areas

  • Social Psychology
  • Applied Psychology
  • Psychology (miscellaneous)

Fingerprint Dive into the research topics of 'Impact of questionnaire format on reliability, validity, and hypothesis testing'. Together they form a unique fingerprint.

Cite this