Director of Research, The Evaluation Center at Western Michigan University
Lori has a Ph.D. in evaluation and more than 20 years of experience in the field of program evaluation. She directs EvaluATE and leads and a variety of evaluation projects at WMU focused on STEM education, health, and higher education initiatives. Dr. Wingate has led numerous webinars and workshops on evaluation in a variety of contexts, including CDC University and the American Evaluation Association Summer Evaluation Institute. She is an associate member of the graduate faculty at WMU.
This is an example project logic model based on a fictional project. The purpose is to better understand what to include in a logic model, and then how to develop evaluation questions based on the logic model. This example is an excerpt from the Evaluation Basics for Non-evaluators webinar. Access slides, recording, handout, and additional resources from bit.ly/mar18-webinar.
Highlights the four main steps of an ATE Evaluation, and provides detailed activities for each step. This example is an excerpt from the Evaluation Basics for Non-evaluators webinar. Access slides, recording, handout, and additional resources from bit.ly/mar18-webinar.
This diagram provides an overview of evaluation responsibilities for the project staff, external evaluator, and combined responsibilities. This example is an excerpt from the Evaluation Basics for Non-evaluators webinar. Access slides, recording, handout, and additional resources from bit.ly/mar18-webinar.
An overview of the NSF Advanced Technological Education program, broken out into funding levels. This is an excerpt from the Evaluation Basics for Non-evaluators webinar. Access slides, recording, handout, and additional resources from bit.ly/mar18-webinar.
e · val · u · a · tion: determination of the value, nature, character, or quality of something or someone*
But what is program evaluation?
Why does the National Science Foundation (NSF) require that the projects they fund be evaluated? How much does it cost? Who can do it? What does a good evaluation plan look like? What will happen? What are you supposed to do with the results?
In this webinar, we’ll answer these and other common questions about program evaluation. This session is for individuals with limited experience with program evaluation, especially two-year college faculty and grants specialists who are planning on submitting proposals to NSF’s Advanced Technological Education program this fall.
This report summarizes data gathered in the 2017 survey of ATE program grantees. Conducted by EvaluATE—the evaluation support center for the ATE program, located at The Evaluation Center at Western Michigan University—this was the 18th annual ATE survey. Included here are findings about funded projects and their activities, accomplishments, and impacts during the 2016 calendar year (2016 fiscal year for budget-related questions).
The purpose of this handbook is to help those who are responsible for organizing, planning, or conducting NVC meetings. It is mainly intended for principal investigators (PIs), but other audiences include center staff, committee chairs, committee members, and others with responsibilities related to NVCs. The NVC Handbook is not intended to establish policy, nor does it necessarily apply to other NSF programs.
New ATE project principal investigators (PIs): When you worked with your evaluator to develop an evaluation plan for your project proposal, you were probably focused on the big picture—how to gather credible and meaningful evidence about the quality and impact of your work. To ensure your evaluation achieves its aims, take these four steps now to make sure your project provides the human resources, time, and information needed for a successful evaluation:
Schedule regular meetings with your evaluator. Regular meetings help ensure that your project’s evaluation receives adequate attention. These exchanges should be in real time—via phone call, web meetings, or face-to-face—not just email. See EvaluATE’s new Communication Plan Checklist for ATE PIs and Evaluators for a list of other communication issues to discuss with your evaluator at the start of a project.
Work with your evaluator to create a project evaluation calendar. This calendar should span the life of your project and include the following:
Due dates for National Science Foundation (NSF) annual reports: You should include your evaluation reports or at least information from the evaluation in these reports. Work backward from their due dates to determine when evaluation reports should be completed. To find out when your annual report is due, go to Research.gov, enter your NSF login information, select “Awards & Reporting,” then “Project Reports.”
Advisory committee meeting dates: You may want your evaluator to attend these meetings to learn more about your project and to communicate directly with committee members.
Project events: Activities such as workshops and outreach events present valuable opportunities to collect data directly from the individuals involved in the project. Make sure your evaluator is aware of them.
Due dates for new proposal submissions: If submitting to NSF again, you will need to include evidence of your current project’s intellectual merit and broader impacts. Working with your evaluator now will ensure you have compelling evidence to support a future submission.
Keep track of what you’re doing and who is involved. Don’t leave these tasks to your evaluator or wait until the last minute. Taking an active—and proactive—role in documenting the project’s work will save you time and result in more accurate information. Your evaluator can then use that information when preparing their reports. Moreover, you will find it immensely useful to have good documentation at your fingertips when preparing your annual NSF report.
Maintain a record of project activities and products—such as conference presentations, trainings, outreach events, competitions, publications—as they are completed. Check out EvaluATE’s project vita as an example.
Create a participant database (or spreadsheet): Everyone who engages with your project should be listed. Record their contact information, role in the project, and pertinent demographic characteristics (such as whether a student is a first-generation college student, a veteran, or part of a group that has been historically underrepresented in STEM). You will probably find several uses for this database, such as for follow-up with participants for evaluation purposes, for outreach, and as evidence of your project’s broader impacts.
An ounce of prevention is worth of pound of cure: Investing time up front to make sure your evaluation is on solid footing will save headaches down the round.
Creating a clear communication plan at the beginning of an evaluation can help project personnel and evaluators avoid confusion, misunderstandings, or uncertainty. The communication plan should be an agreement between the project’s principal investigator and the evaluator, and followed by members of their respective teams. This checklist highlights the decisions that need to made when developing a clear communication plan.
Evaluation is an important element of an ATE proposals. EvaluATE has developed several resources to help you develop your evaluation plans and integrate them into your ATE proposals. This video highlights a few of them—these and more can be accessed from the links below the video.