Development and evaluation of a simulation-based pediatric emergency medicine curriculum

PURPOSE: The infrequency of severe childhood illness limits opportunities for emergency medicine (EM) providers to learn from real-world experience. Simulation offers an evidence-based educational approach to develop and practice clinical skills. METHOD: This was a two-phase, randomized trial with a wait-list control condition. The development phase (2005-2006) involved systematic curriculum and rating checklist creation, producing a six-case, simulation-based curriculum linked to three evaluation cases.In the validation phase (2006-2007), the authors randomized 69 residents from two EM residencies to either an intervention group that received the curriculum one month before the first assessment of all participants or a wait-list control group that received the identical curriculum three months later. A final assessment of all residents followed one month after that. Two raters evaluated all residents. Primary outcome measures are percentages of items completed correctly. The authors assessed rater agreement using intraclass correlation (ICC) and compared group performance using mixed-model analysis of variance. RESULTS: ICCs surpassed 0.78. The instructional intervention produced a statistically significant effect for two of three evaluation cases for the validation phase of the study, a case × occasion interaction. Training year was significantly associated with better performance. In a multivariate analysis, training year and session correlated with score, but study group did not. CONCLUSIONS: A one-day, simulation-based pediatric EM curriculum produced limited results. The evaluation approach is reasonable and reproducible for the population studied. Instructional dose strength and factors may have limited curriculum effectiveness. Focused, frequent, and effortful instructional interventions are necessary to achieve substantial performance improvements.