Issues in the evaluation of a multi-site federal systems change initiative

This article describes an attempt to evaluate the transition systems change initiative under trying but typical circumstances. Given a large number of geographically dispersed sites, each with quite different goals, activities, and impacts, the evaluators were charged with the task of describing and assessing the impact of the entire initiative. Though each of the funded sites were conducting their own evaluations, the scope and focus of these evaluations varied greatly. The evaluation described in this paper was designed to assess the common impacts and unique accomplishments of this array of projects, making maximum use of existing data and minimizing the burden on project directors for new data collection. The design was participatory, involving project directors in planning the evaluation, reporting and reflecting on their own project impact, and validating the evaluation findings. It is hoped that the design of the evaluation might be applicable to similar large scale, multi-site initiatives.