Updated July 2019
This checklist provides information on what should be included in evaluation plans for proposals to the National Science Foundation’s (NSF) Advanced Technological Education (ATE) program. Grant seekers should carefully read the most recent ATE program solicitation (http://bit.ly/nsf-ate) for details about the program and proposal submission requirements.
Evaluation Plan
ATE proposals must include a subsection titled “Evaluation Plan” within the 15-page project description. EvaluATE recommends dedicating one to two pages to the evaluation plan and including the following five elements:
1. Evaluator
□ Identify the project’s evaluator by name and organization.
□ Briefly describe the evaluator’s qualifications, including their experience evaluating STEM education programs.
□ Refer to the evaluator’s biosketch and letter of collaboration and include these as supplementary documents.
□ If the evaluator is an employee of the project’s host institution, explain how the evaluator is independent from the project (they should not work in the same department or be a supervisor or supervisee of project personnel).
If the project’s host institution has a policy that prohibits selecting an evaluator at the proposal stage:
□ Explain the institutional policy that does not allow for selection of an evaluator prior to funding.
□ Describe how an evaluator will be selected after the award is made.
2. Evaluation Questions
□ List key questions—ideally, about three to seven—that the evaluation will address.
□ Include questions about both project implementation (what the project does) and outcomes (what changes it brings about).
□ Ensure that the questions align with the project’s goals and activities as described in the proposal.
□ Ensure that the questions address the project’s intellectual merit (contributions to advancing knowledge) and broader impact (contributions to the betterment of society).
3. Data
Indicators
□ Identify what information will be used to answer each evaluation question (i.e., what will be measured).
Data Collection Methods and Sources
□ Identify how the information will be gathered and from what sources.
□ If relevant, explain sampling and use of comparison or control groups.
□ If using existing data collection instruments, include citations and justify their use.
Analysis
□ Identify the procedures that will be used to summarize quantitative and qualitative data (e.g., descriptive statistics, inferential tests, regression, deductive or inductive coding).
Interpretation
□ Explain how findings will be interpreted to answer the evaluation questions (e.g., compare results with baseline or needs assessment data, with targets/benchmarks, or between groups; use rubrics; engage stakeholders).
4. Communication and Use of Results
□ Identify how evaluation results will be communicated to the project team (e.g., interim and annual reporting, presentations, feedback sessions).
□ Note the frequency with which the evaluator will communicate with the project team (e.g., quarterly meetings or monthly conference calls).
□ Describe how evaluation results will be shared with external audiences who could benefit from the information (e.g., publications, conference presentations, newsletters).
□ Identify how the evaluation results will be used to improve the project.
5. Timeline
□ Identify when important evaluation activities—such as data collection, reporting, and dissemination of results—will take place. (This information may be included in the evaluation section or integrated into the overall project timeline.)
Evaluation in Other Proposal Sections
In addition to being placed in the Evaluation Plan section, information related to evaluation should appear in the following sections of the proposal:
Results from Prior NSF Support
If the ATE proposal’s principal investigator (PI) or co-PI has received NSF funding within the past five years, the current proposal’s project description must begin with a subsection titled “Results from Prior NSF Support.” In this section, describe the specific achievements and outcomes of previously funded NSF projects related to the NSF review criteria of Intellectual Merit and Broader Impacts, with supporting evidence from the project’s evaluation, if available.
Budget and Budget Justification
The ATE program solicitation states that “funds to support an evaluator independent of the project must be requested.” The evaluation budget should be consistent with the scope of the evaluation effort. Unless the evaluator is employed by the project’s home institution, include the costs as “consultant services” or a “subaward” (there isn’t a rule for which to use). Different requirements apply for each:
Consultant
|
Subaward
|
Data Management Plan
Data management plans are required for all NSF proposals. These documents may be up to two pages long and are uploaded separately from the proposal’s project description. They describe the data and other materials that will be generated by the project and how that information will be shared and preserved. The plan should address all data collected and products generated by the project, including those generated by the evaluation.
References Cited
References to evaluation literature help show how the evaluation is grounded in and building on current knowledge and practice. If a specific evaluation approach or instrument will be used, provide citations to support its use in the proposed project.
Logic Model
Logic models are not required for ATE proposals, but they are useful for providing an overview of a project and showing how evaluation questions align with project activities and intended outcomes. A logic model should not exceed one page. Do not include a logic model as a separate supplementary document—the ATE program allows only specific types of supplementary documents.
Resources
Evaluator
-
- Evaluator Biosketch Template
- Guide to Finding and Selecting an Evaluator for ATE Proposals
- Recommended text for letters of collaboration: “If the proposal submitted by [full name of the principal investigator], titled [proposal title], is selected for funding by NSF, it is my intent to collaborate and/or commit resources as detailed in the project description.” See http://bit.ly/pappg-coll
- What Should I Do if My College’s Procurement Office Won’t Let Me Name an Evaluator in My Proposal? [article]
- Evaluation Procurement: Regulations, Rules, and Red Tape…Oh My! [blog]
Evaluation Questions
-
- Evaluation Questions Checklist
- Intellectual Merit and Broader Impacts: Identifying Your Project’s Achievements and Supporting Evidence [blog]
- Definitions of Intellectual Merit and Broader Impact (from NSF Proposal and Award Policies and Procedures Guide)
Data
Communication and Use of Results
For guidance on how to integrate the elements above into a concise evaluation plan, see ATE Proposal Evaluation Plan Template
Results from Prior NSF Support
Data Management Plan
-
- NSF Proposal and Award Policies and Procedures Guide (see section on data management plans)
- ATE Central—Data Management Planning
Logic Model
Current and Pending Support
-
- Current and Pending Support Template (Word download)