Issues in the Evaluation of a Multi-Site Federal System Change Initiative

In 1991, the Office of Special Education and Rehabilitation Services (OSERS) authorized, under the Individuals with Disabilities Act (IDEA), a specific grants program. The State Systems for Transitions Services for Youth with Disabilities Program made available competitive 5-year grants to individual states for establishing responsive state systems that address the school-to-work transition needs of youth with disabilities. This paper describes the evaluation-design process for assessing the effectiveness of the 30 funded sites. The evaluation focused on three areas of system change--youth and family engagement in transition, professional skills and knowledge, and agency-community collaboration. Data were gathered from a review of strategies, evidence of change, and supports/constraints; annual interviews with project directors; and collection of contextual information. Preliminary analysis identified 10 initial influences of the systems-change projects: (1) multiple strategies for change; (2) increased numbers of transition personnel within state education agencies; (3) increased responsiveness of interagency mechanisms; (4) expanded state and local professional-development opportunities; (5) increased availability and quality of transition services; (6) increased parent access to information and training; (7) increased participation of youth with disabilities in state school-to-work initiatives; (8) improved state-level evaluation systems; (9) increased participation of underrepresented groups; and (10) increased availability and access to information on transition policies, programs, and practices. Appendices contain the conceptual framework; sample strategies, evidence of change, and supports/constraints; the project directors' interview protocol; a sample interview; data sources; an activity matrix; a sample state description; and the evaluation timeline.