Asymptotically Normally Distributed Person Fit Indices for Detecting Spuriously High Scores on Difficult Items

Yan Xia, Yi Zheng

Research output: Contribution to journalArticle

Abstract

Snijders developed a family of person fit indices that asymptotically follow the standard normal distribution, when the ability parameter is estimated. So far, l* z, U*, W*, ECI2* z, and ECI4* z from this family have been proposed in previous literature. One common property shared by l* z, U*, and W* (also ECI2* z, and ECI4* z in some specific conditions) is that they employ symmetric weight functions and thus identify spurious scores on both easy and difficult items in the same manner. However, when the purpose is to detect only the spuriously high scores on difficult items, such as cheating, guessing, and having item preknowledge, using symmetric weight functions may jeopardize the detection rates of the target aberrant response patterns. By specifying two types of asymmetric weight functions, this study proposes SHa(λ)* (λ = 1/2 or 1) and SHb(β)* (β = 2 or 3) based on Snijders’s framework to specifically detect spuriously high scores on difficult items. Two simulation studies were carried out to investigate the Type I error rates and empirical power of SHa(λ)* and SHb(β)*, compared with l* z, U*, W*, ECI2* z, and ECI4* z. The empirical results demonstrated satisfactory performance of the proposed indices. Recommendations were also made on the choice of different person fit indices based on specific purposes.

Original languageEnglish (US)
Pages (from-to)343-358
Number of pages16
JournalApplied Psychological Measurement
Volume42
Issue number5
DOIs
StatePublished - Jul 1 2018
Externally publishedYes

Fingerprint

Weights and Measures
human being
common property
Normal Distribution
simulation
ability
performance
literature

Keywords

  • asymptotically standard normal distribution
  • item response theory
  • person fit statistic

ASJC Scopus subject areas

  • Social Sciences (miscellaneous)
  • Psychology (miscellaneous)

Cite this

Asymptotically Normally Distributed Person Fit Indices for Detecting Spuriously High Scores on Difficult Items. / Xia, Yan; Zheng, Yi.

In: Applied Psychological Measurement, Vol. 42, No. 5, 01.07.2018, p. 343-358.

Research output: Contribution to journalArticle

@article{faaa6892e27849df9ce15cc3d18b51ee,
title = "Asymptotically Normally Distributed Person Fit Indices for Detecting Spuriously High Scores on Difficult Items",
abstract = "Snijders developed a family of person fit indices that asymptotically follow the standard normal distribution, when the ability parameter is estimated. So far, l* z, U*, W*, ECI2* z, and ECI4* z from this family have been proposed in previous literature. One common property shared by l* z, U*, and W* (also ECI2* z, and ECI4* z in some specific conditions) is that they employ symmetric weight functions and thus identify spurious scores on both easy and difficult items in the same manner. However, when the purpose is to detect only the spuriously high scores on difficult items, such as cheating, guessing, and having item preknowledge, using symmetric weight functions may jeopardize the detection rates of the target aberrant response patterns. By specifying two types of asymmetric weight functions, this study proposes SHa(λ)* (λ = 1/2 or 1) and SHb(β)* (β = 2 or 3) based on Snijders’s framework to specifically detect spuriously high scores on difficult items. Two simulation studies were carried out to investigate the Type I error rates and empirical power of SHa(λ)* and SHb(β)*, compared with l* z, U*, W*, ECI2* z, and ECI4* z. The empirical results demonstrated satisfactory performance of the proposed indices. Recommendations were also made on the choice of different person fit indices based on specific purposes.",
keywords = "asymptotically standard normal distribution, item response theory, person fit statistic",
author = "Yan Xia and Yi Zheng",
year = "2018",
month = "7",
day = "1",
doi = "10.1177/0146621617730391",
language = "English (US)",
volume = "42",
pages = "343--358",
journal = "Applied Psychological Measurement",
issn = "0146-6216",
publisher = "SAGE Publications Inc.",
number = "5",

}

TY - JOUR

T1 - Asymptotically Normally Distributed Person Fit Indices for Detecting Spuriously High Scores on Difficult Items

AU - Xia, Yan

AU - Zheng, Yi

PY - 2018/7/1

Y1 - 2018/7/1

N2 - Snijders developed a family of person fit indices that asymptotically follow the standard normal distribution, when the ability parameter is estimated. So far, l* z, U*, W*, ECI2* z, and ECI4* z from this family have been proposed in previous literature. One common property shared by l* z, U*, and W* (also ECI2* z, and ECI4* z in some specific conditions) is that they employ symmetric weight functions and thus identify spurious scores on both easy and difficult items in the same manner. However, when the purpose is to detect only the spuriously high scores on difficult items, such as cheating, guessing, and having item preknowledge, using symmetric weight functions may jeopardize the detection rates of the target aberrant response patterns. By specifying two types of asymmetric weight functions, this study proposes SHa(λ)* (λ = 1/2 or 1) and SHb(β)* (β = 2 or 3) based on Snijders’s framework to specifically detect spuriously high scores on difficult items. Two simulation studies were carried out to investigate the Type I error rates and empirical power of SHa(λ)* and SHb(β)*, compared with l* z, U*, W*, ECI2* z, and ECI4* z. The empirical results demonstrated satisfactory performance of the proposed indices. Recommendations were also made on the choice of different person fit indices based on specific purposes.

AB - Snijders developed a family of person fit indices that asymptotically follow the standard normal distribution, when the ability parameter is estimated. So far, l* z, U*, W*, ECI2* z, and ECI4* z from this family have been proposed in previous literature. One common property shared by l* z, U*, and W* (also ECI2* z, and ECI4* z in some specific conditions) is that they employ symmetric weight functions and thus identify spurious scores on both easy and difficult items in the same manner. However, when the purpose is to detect only the spuriously high scores on difficult items, such as cheating, guessing, and having item preknowledge, using symmetric weight functions may jeopardize the detection rates of the target aberrant response patterns. By specifying two types of asymmetric weight functions, this study proposes SHa(λ)* (λ = 1/2 or 1) and SHb(β)* (β = 2 or 3) based on Snijders’s framework to specifically detect spuriously high scores on difficult items. Two simulation studies were carried out to investigate the Type I error rates and empirical power of SHa(λ)* and SHb(β)*, compared with l* z, U*, W*, ECI2* z, and ECI4* z. The empirical results demonstrated satisfactory performance of the proposed indices. Recommendations were also made on the choice of different person fit indices based on specific purposes.

KW - asymptotically standard normal distribution

KW - item response theory

KW - person fit statistic

UR - http://www.scopus.com/inward/record.url?scp=85041912445&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85041912445&partnerID=8YFLogxK

U2 - 10.1177/0146621617730391

DO - 10.1177/0146621617730391

M3 - Article

AN - SCOPUS:85041912445

VL - 42

SP - 343

EP - 358

JO - Applied Psychological Measurement

JF - Applied Psychological Measurement

SN - 0146-6216

IS - 5

ER -