As an ATE project, you and your team collect a lot of data: You complete the annual monitoring survey, you work with your evaluator to measure outcomes, you may even track your participants longitudinally in order to learn how they integrate their experiences into their lives. As overwhelming as it may seem at times to manage all the data collection logistics and report writing, these data are important to telling the story of your project and the ATE program. Developing a culture of evaluation in your project and your organization can help to meaningfully put these data to use.
Fostering a culture of evaluation in your project means that evaluation practices are not disconnected from program planning, implementation, and reporting. You’re thinking of evaluation in planning project activities and looking for ways to use data to reflect on and improve your work. During implementation, you consult your evaluator regularly so that you can hear what they’re learning from the data collection, and ensure that they know what’s new in the project. And at analysis and reporting times, you’re ensuring that the right people are thinking about how to use the evaluation findings to make improvements and demonstrate your project’s value to important stakeholder audiences. You and your team are reflecting on how the evaluation went and what can be improved. In a project that has an “evaluation culture,” evaluators are partners, collecting important information to inform decision making.
A great example of evaluators-as partners came from an NSF PI who shared that he regularly talks with his evaluator, peer-to-peer, about the state of the field, not just about his particular project. He wants to now what his evaluator is learning about practice in the field from other projects, workshops, conferences and meetings, and he uses these insights to help him reflect on his own work.
Fellow grant writers: Do you ever stop and ask yourselves, “Why do we write grants?” Do you actually enjoy herding cats, pulling teeth, or the inevitable stress of a looming proposal deadline? I hope not. Then what is the driver? We shouldn’t just write a grant to simply get funded or to earn prestige for our colleges. Those benefits may be motivators, but we should write to get positive results for the faculty, students, and institutions involved. And we should be able to evaluate those results in useful and meaningful ways so that we can identify ways to improve and demonstrate the project’s value.
Evaluation isn’t just about satisfying a promise or meeting a requirement to gather and report data, it’s about gathering meaningful data that can be utilized to determine the effectiveness of an activity and the impact of a project. When developing a grant proposal, one often starts with the goals, then thinks of the objectives and then plans the activities, hoping that in the end, the evaluation data will prove that the goals were met and the project was a success. That is putting a lot of faith in “hope.” I find it more promising to begin with the end in mind from an evaluation perspective: What is the positive change that we hope to achieve and how will it be evidenced? What does success mean? How can we tell? When will we know? And, how can we get participants to provide the information we will need for the evaluation?
The role of a grant writer is too often like that of a quilt maker—sections of the grant solicitation are delegated to different members of the institution with the evaluation section often outsourced to a third-party evaluator. Each party submits their content, then the grant writer scrambles to patch it all together. Now instead of a quilt, consider the construction of a tapestry. Instead of chunks of material stitched together in independent sections, each thread is carefully woven in a thoughtful way to create a larger, more cohesive overall design. It is important that the entire development team work together to fully understand each aspect of the proposal in order to collaboratively develop a coherent plan to obtain the desired outcomes. The project workplan, budget, and evaluation components should not be designed or executed independently—they occur simultaneously and are dependent upon each other.
I encourage you to think like an evaluator as you develop your proposals. Prepare yourself and challenge your team to be able to justify the value of each goal, objective, and activity and be able to explain how that value will be measured. If at all possible, involve your external or internal evaluator early on in the proposal development. The better the evaluator understands your overall concept and activities, the better he or she can tailor the evaluation plan to derive the desired results. A strong workplan and evaluation plan will help proposal reviewers connect the dots and see the potential of your proposal. It will also serve as a roadmap to success for your project implementation team.