Assessing the self-reported honesty threshold in adolescent epidemiological research: comparing supervised machine learning and inferential statistical techniques

Janaka V. Kosgolla, Douglas C. Smith, Shahana Begum, Crystal A. Reinhart

Research output: Contribution to journalArticlepeer-review

Abstract

Background: Epidemiological surveys offer essential data on adolescent substance use. Nevertheless, the precision of these self-report-based surveys often faces mistrust from researchers and the public. We evaluate the efficacy of a direct method to assess data quality by asking adolescents if they were honest. The main goal of our study was to assess the accuracy of a self-report honesty item and designate an optimal threshold for it, allowing us to better account for its impact on point estimates. Methods: The participants were from the 2020 Illinois Youth Survey, a self-report school-based survey. We divided the primary dataset into subsets based on responses to an honesty item. Then, for each dataset, we examined two distinct data analysis methodologies: supervised machine learning, using the random forest algorithm, and a conventional inferential statistical method, logistic regression. We evaluated item thresholds from both analyses, investigating probable relationships with reported fake drug use, social desirability biases, and missingness in the datasets. Results: The study results corroborate the appropriateness and reliability of the honesty item and its corresponding threshold. These contain the agreeing honesty thresholds determined in both data analyses, the identified association between reported fake drug use and lower honesty scores, increased missingness and lower honesty, and the determined link between the social desirability bias and honesty threshold. Conclusions: Confirming the honesty threshold via missing data analysis also strengthens these collective findings, emphasizing our methodology’s and findings’ robustness. Researchers are encouraged to use self-report honesty items in epidemiological research. This will permit the modeling of accurate point estimates by addressing questionable reporting.

Original languageEnglish (US)
Article number210
JournalBMC Medical Research Methodology
Volume23
Issue number1
DOIs
StatePublished - Dec 2023

Keywords

  • Adolescents
  • Epidemiological surveys
  • Machine learning
  • Response validity
  • Self-reported honesty
  • Substance use

ASJC Scopus subject areas

  • Health Informatics
  • Epidemiology

Fingerprint

Dive into the research topics of 'Assessing the self-reported honesty threshold in adolescent epidemiological research: comparing supervised machine learning and inferential statistical techniques'. Together they form a unique fingerprint.

Cite this