Archive: guidelines

Newsletter: What makes a good evaluation section of a proposal?

Posted on July 1, 2013 by  in Newsletter - ()

Principal Research Scientist, Education Development Center, Inc.

As a program officer, I read hundreds of proposals for different NSF programs and I saw many different approaches to writing a proposal evaluation section. From my vantage point, here are a few tips that may help to ensure that your evaluation section shines.

First, make sure to involve your evaluator in writing the proposal’s evaluation section. Program officers and reviewers can tell when an evaluation section was written without the consultation of an evaluator. This makes them think you aren’t integrating evaluation into your project planning.

Don’t just call an evaluator a couple weeks before the proposal is due! A strong evaluation section comes from a thoughtful, robust, tailored evaluation plan. This takes collaboration with an evaluator! Get them on board early and talk with them often as you develop your proposal. They can help you develop measureable objectives, add insight to proposal organization, and, of course, work with you to develop an appropriate evaluation plan.

Reviewers and program officers look to see that the evaluator understands the project. This can be done using a logic model or in a paragraph that justifies the evaluation design, based on the proposed project design. The evaluation section should also connect the project objectives and targeted outcomes to evaluation questions, data collection methods and analysis, and dissemination plans. This can be done in a matrix format, which helps the reader to see clearly which data will answer which evaluation question and how these are connected to the objectives of the project.

A strong evaluation plan shows that the evaluator and the project team are in synch and working together, applies a rigorous design and reasonable data collection methods, and answers important questions that will help to demonstrate the value of the project and surface areas for improvement.

Newsletter: What does the switch from FastLane to Research.gov mean for ATE annual reporting?

Posted on April 1, 2013 by  in Newsletter - ()

Director of Research, The Evaluation Center at Western Michigan University

Along with the change from FastLane to Research.gov, there are a few changes in the reporting categories. Additionally, public Project Outcomes Reports are now required in addition to Final Reports 90 days after grant expires.

ANNUAL REPORTS

The annual report categories have largely remained the same, but there are a few noteworthy changes. The Participants section has not changed—this is where you identify the individuals, organizations, and other collaborators that have contributed to your grant work. A new Accomplishments section replaces the old “Activities and Findings” component. This is where most of the new requirements are found. In addition to identifying the project’s major goals, PIs must provide information for at least one of the following categories: major activities, specific objectives; significant results (including findings, developments, or conclusions) and key outcomes or other achievements. I recommend reporting your evaluation results under “key outcomes.” The Products section of the report is very similar to what was included in the FastLane system, but now in addition to publications, websites, and other products, there are separate areas to identify (a) technologies or techniques and (b) inventions, patent applications, and/or other licenses. The former “Conference Proceedings” section is now subsumed in this category as well. A new Impact section replaces what was formerly called “Contributions.” In addition to contributions to the principal discipline, other disciplines, human resources, infrastructure resources, and other aspects of public welfare (now labeled “beyond science and technology”), PIs are now asked to also report on technology transfer and identify significant problems or changes in the project.

PROJECT OUTCOMES REPORTS

Project Outcomes Reports are 200-800 word summaries of projects and their outcomes. In particular, PIs should address results that address NSF’s intellectual merit and broader impacts review criteria. For intellectual merit, you should address how the project has advanced knowledge and understanding around technician education and/or how the project has been especially creative, original, or transformative. Here you can refer to how you have described your disciplinary contributions in the Impact section of your annual report. As for broader impacts, this is your opportunity to describe your impact on students; what you have done to broaden participation of underrepresented groups; how you have enhanced infrastructure for technological education and research through facilities, instrumentation, networks, and partnerships; and/or other ways your grant-funded work has benefitted society.

To learn more about Research.gov, annual reporting requirements, and project outcomes reports, go to 1.usa.gov/16NJqXr.

To view examples of ATE Project Outcomes Reports, go to 1.usa.gov/13j3Us7, then click on the box for “Show Only Awards with Project Outcomes Reports” and in the box for “Program,” type “Advanced Technological Education.”