This tool can help users to ensure that they have developed a viable plan for collecting all the data necessary to answer each evaluation question and that all data collected will serve a specific, intended purpose.
The part of your proposal’s evaluation plan that reviewers will probably scrutinize most closely is the data collection plan. Given that the evaluation section of a proposal is typically just 1-2 pages, you have minimal space to communicate a clear plan for gathering evidence of your project’s quality and impact. An efficient way to convey this information is in a matrix format. To help with this task, we’ve created a Data Collection Planning Matrix, available from (bit.ly/data-matrix).
This tool prompts the user to specify the evaluation questions that will serve as the foundation for the evaluation; what indicators1 will be used to answer each evaluation question; how data for each indicator will be collected, from what sources, by whom, and when; and how the data will be analyzed. (The document includes definitions for each of these components to support shared understandings among members of the proposal development team.) Including details about data collection in your proposal shows reviewers that you have been thoughtful and strategic in determining how you will build a body of evidence about the effectiveness and quality of your NSF-funded work. The value of putting this information in a matrix format is that it ensures you have a clear plan for gathering data that will enable you to fully address all the evaluation questions and, conversely, that all the data you plan to collect will serve a specific purpose.
A good rule of thumb is to develop at least one overarching evaluation question for each main element of a project logic model (i.e., activities, outputs, and short-, mid-, and long-term outcomes). Although not required for ATE program proposals, logic models are an efficient way to convey how your project’s activities and products will lead to intended outcomes. The evaluation’s data collection plan should align clearly with your project’s activities and goals, whether you use a logic model or not. If you are interested in developing a logic model for your project and want to learn more, see our ATE Logic Model Template at (bit.ly/ate-logic).
If you have questions about the data collection planning matrix or logic model template or suggestions for improving it, let us know: email us at firstname.lastname@example.org.
1 For more on indicators and how to select ones that will serve your evaluation well, see Goldie MacDonald’s checklist, Criteria for Selection of High-Performing Indicators, available from (bit.ly/indicator-eval).
and other resources to assist in proposal development and evaluation planning
To assist ATE proposers navigate the intersection of proposal development and evaluation planning, EvaluATE developed an Evaluation Planning Checklist for ATE Proposals. There is more to addressing evaluation in your proposal than including a section on evaluation. Information pertinent to your evaluation should also be evident in your project summary, references, results of prior NSF support (first part of the project description for those who’ve received NSF funding before), budget and budget justification, and supplementary documents. Organized by proposal component, the checklist provides details about what you need to know and do in order to integrate evaluation into your proposal. This checklist was originally released last fall. Since then, it has undergone revisions based on feedback from members of the ATE community.
We also recommend you read the advice of Elizabeth Teles, former ATE program co-lead and member of EvaluATE’s National Visiting Committee. You can access Dr. Teles’s 10 Helpful Hints and 10 Fatal Flaws: Writing Better Evaluation Sections in Your Proposals.
Another resource proposers may find useful is EvaluATE’s Logic Model Template. Preformatted with editable text boxes, this one-page document is designed so that you can quickly and easily modify it to suit your own needs. Logic models are useful for project development, evaluation planning, and monitoring progress.