TY - GEN
T1 - Superficial Code-guise
T2 - 52nd ACM Technical Symposium on Computer Science Education, SIGCSE 2021
AU - Fowler, Max
AU - Zilles, Craig
N1 - Publisher Copyright:
© 2021 ACM.
PY - 2021/3/3
Y1 - 2021/3/3
N2 - Assessing student performance on programming questions is important for introductory computer science courses, both for student learning and for ensuring students demonstrate competence. Part of being a competent programmer includes the ability to transfer learning from solved to analogous problems. Additionally, particularly in computer-based and online assessment, mitigating cheating efforts is another important consideration. One way to mitigate cheating is by randomly selecting from large pools of equivalent questions. In order to produce large pools of questions quickly, we used a permutation strategy to rapidly make new question variants by altering existing questions' surface features. In this work, we present the results of our first set of surface feature permuted questions in an introductory Python course. We find surface feature permutations to be an effective way to produce questions of a similar difficulty to other new questions for students while mitigating potential cheating. However, we also see permutations expose potential student knowledge fragility and transfer concerns, as performance on permutations of homework questions is not strictly better than performance on questions that are entirely new on assessments
AB - Assessing student performance on programming questions is important for introductory computer science courses, both for student learning and for ensuring students demonstrate competence. Part of being a competent programmer includes the ability to transfer learning from solved to analogous problems. Additionally, particularly in computer-based and online assessment, mitigating cheating efforts is another important consideration. One way to mitigate cheating is by randomly selecting from large pools of equivalent questions. In order to produce large pools of questions quickly, we used a permutation strategy to rapidly make new question variants by altering existing questions' surface features. In this work, we present the results of our first set of surface feature permuted questions in an introductory Python course. We find surface feature permutations to be an effective way to produce questions of a similar difficulty to other new questions for students while mitigating potential cheating. However, we also see permutations expose potential student knowledge fragility and transfer concerns, as performance on permutations of homework questions is not strictly better than performance on questions that are entirely new on assessments
KW - CS1
KW - introductory computer science
KW - online assessment
KW - programming surface features
UR - http://www.scopus.com/inward/record.url?scp=85103342134&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85103342134&partnerID=8YFLogxK
U2 - 10.1145/3408877.3432413
DO - 10.1145/3408877.3432413
M3 - Conference contribution
AN - SCOPUS:85103342134
T3 - SIGCSE 2021 - Proceedings of the 52nd ACM Technical Symposium on Computer Science Education
SP - 3
EP - 9
BT - SIGCSE 2021 - Proceedings of the 52nd ACM Technical Symposium on Computer Science Education
PB - Association for Computing Machinery, Inc
Y2 - 13 March 2021 through 20 March 2021
ER -