This is the pdf version of our Spring 2015 Newsletter
This report was prepared as an analysis of the first annual status report. It was meant to help understand the ATE program, and make preparations for upcoming surveys and site visits.
The development of high-quality materials to support the delivery of curricula and promote educational change is critical to the goals of the ATE program. This issue-oriented paper was written to provide a framework for the development of high-quality curriculum materials that can serve as a guide for those who are considering developing a proposal to the ATE program, and to guide NSF in the review of proposals and evaluation of funded projects and centers.
This brief focuses on project/center evaluation and is divided into 4 sections. This section, Section 1, provides an overview of ATE expectations for evaluation and principal investigators’ responses that describe how they meet those requirements—who conducts the evaluations, how much money is spent on evaluations, and the extent to which these evaluations vary by characteristics such as the type of grant and type of evaluator conducting the evaluation. Section 2 describes PI perceptions of the utility of their evaluations and the extent to which PI perceptions of utility are related to the evaluation characteristics described in Section 1. Section 3 focuses on the activities of external evaluators — PI satisfaction with these evaluators, the relationship between PI ratings and standards for sound program evaluations, whether the PIs view their evaluations as meeting ATE intellectual merit requirements, and PIs’ characterizations of the attributes of their external evaluators. Section 4 draws together findings reported in Sections 1 to 3 to identify strengths and weaknesses of project-level evaluations and to suggest changes that appear likely to improve on current evaluation practices.
This study analyzes project-level evaluation practices occurring in the Advanced Technological Education program of the National Science Foundation. Of special interest in this study were factors thought to affect the quality and utility of evaluations such as the cost of evaluations, who engaged in evaluation planning, and the use of external evaluators. The ATE program requires project-level evaluations and provides guidelines regarding what evaluations can and should do. The report closes with a discussion of discrepancies between expectations and project level actions and the apparent strengths and weaknesses of project evaluations. Suggestions are offered on how to improve these evaluation practices.