Initial Validation of the Cybersecurity Concept Inventory: Pilot Testing and Expert Review

Spencer Offenberger, Geoffrey L. Herman, Peter Peterson, Alan T. Sherman, Enis Golaszewski, Travis Scheponik, Linda Oliva

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We analyze expert review and student performance data to evaluate the validity of the Cybersecurity Concept Inventory (CCI) for assessing student knowledge of core cybersecurity concepts after a first course on the topic. A panel of 12 experts in cybersecurity reviewed the CCI, and 142 students from six different institutions took the CCI as a pilot test. The panel reviewed each item of the CCI and the overwhelming majority rated every item as measuring appropriate cybersecurity knowledge. We administered the CCI to students taking a first cybersecurity course either online or proctored by the course instructor. We applied classical test theory to evaluate the quality of the CCI. This evaluation showed that the CCI is sufficiently reliable for measuring student knowledge of cybersecurity and that the CCI may be too difficult as a whole. We describe the results of the expert review and the pilot test and provide recommendations for the continued improvement of the CCI.

Original languageEnglish (US)
Title of host publication2019 IEEE Frontiers in Education Conference, FIE 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728117461
DOIs
StatePublished - Oct 2019
Event49th IEEE Frontiers in Education Conference, FIE 2019 - Covington, United States
Duration: Oct 16 2019Oct 19 2019

Publication series

NameProceedings - Frontiers in Education Conference, FIE
Volume2019-October
ISSN (Print)1539-4565

Conference

Conference49th IEEE Frontiers in Education Conference, FIE 2019
CountryUnited States
CityCovington
Period10/16/1910/19/19

Keywords

  • Cybersecurity Assessment Tools (CATS) Project
  • Cybersecurity Concept Inventory (CCI)
  • Cybersecurity education
  • assessment validation

ASJC Scopus subject areas

  • Software
  • Education
  • Computer Science Applications

Fingerprint Dive into the research topics of 'Initial Validation of the Cybersecurity Concept Inventory: Pilot Testing and Expert Review'. Together they form a unique fingerprint.

  • Cite this

    Offenberger, S., Herman, G. L., Peterson, P., Sherman, A. T., Golaszewski, E., Scheponik, T., & Oliva, L. (2019). Initial Validation of the Cybersecurity Concept Inventory: Pilot Testing and Expert Review. In 2019 IEEE Frontiers in Education Conference, FIE 2019 [9028652] (Proceedings - Frontiers in Education Conference, FIE; Vol. 2019-October). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/FIE43999.2019.9028652