Archive: grants

Blog: Grant Evaluation: What Every PI Should Know and Do*

Posted on June 3, 2019 by  in Blog ()

Luka Partners LLC

Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

A number of years ago, the typical Advanced Technological Education (ATE) Principal Investigator (PI) deemed evaluation a necessary evil. As a PI, I recall struggling even to find an evaluator who appeared to have reasonable credentials. I viewed evaluation as something you had to have in a proposal to get funded.

Having transitioned from the PI role to being an evaluator myself, I now appreciate how evaluation can add value to a project. I also know a lot more about how to find an evaluator and negotiate the terms of the evaluation contract.

Today, PIs typically identify evaluators through networking and sometimes use evaluator directories, such as the one maintained by EvaluATE at ATE Central. You can call colleagues and ask them to identify someone they trust and can recommend with confidence. If you don’t know anyone yet, start your networking by contacting an ATE center PI using the map at atecentral.net. Do this at least three months before the proposal submission date (i.e., now). When you approach an evaluator, ask for a résumé, references, and a work sample or two. Review their qualifications to be sure the proposal’s reviewers will perceive them as a credentialed evaluator.

Second, here is an important question many PIs ask: “Once you have identified the evaluator, can you expect them to write the evaluation section of your proposal for free?” The answer is (usually) yes. Just remember: Naming an individual in your proposal and engaging that person in proposal development reflects your commitment to enter into a contract with them if your proposal is funded. (An important caveat: Many community colleges’ procurement rules require a competition or bid process for evaluation services. That may affect your ability to commit to the evaluator should the proposal be funded. Have a frank discussion about this.)

Although there is a limit to what evaluators can or should do for free at the proposal stage, you should expect more than a boilerplate evaluation plan (provided you’ve allowed enough time for a thoughtful one). You want someone who will take a look at your goals and objectives and describe in 1 to 1.25 pages the approach for this project’s evaluation. This will serve you better than modifying their “standard language,” if they offer it, yourself. Once the proposal is funded, their first deliverable will be the complete evaluation plan; you generally won’t need that level of detail at the proposal stage.

Now that you have a handshake agreement with your selected evaluator, make it clear you need the draft evaluation section by a certain deadline — say, a month before the proposal due date. You do not have to discuss detailed contractual terms prior to the proposal being funded, but you do have to establish the evaluation budget and the evaluator’s daily rate, for your budget and budget justification. Establishing this rate requires a frank discussion about fees.

Communication in this process is key. Check out EvaluATE’s webinar, “Getting Everyone on the Same Page,” practical strategies for evaluator-stakeholder communication.

Once your proposal has been funded, you get to hammer out a real statement of work with your evaluator and set up a contract for the project. Then the real work begins.

*This blog is a reprint of an article from an EvaluATE newsletter published in summer 2012.

Keywords: evaluators, find evaluator, proposal, evaluation, evaluation proposal

Blog: Utilizing Your Institutional Research Office Resources When Writing a Grant Application

Posted on March 20, 2018 by , in Blog ()
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Deborah Douma
Dean, Grants and Federal Programs, Pensacola State College
Michael Johnston
Director of Institutional Research, Pensacola State College

There are a number of guiding questions that must be answered to develop a successful grant project evaluation plan. The answers to these questions also provide guidance to demonstrate need and develop ambitious, yet attainable, objectives. Data does not exist in a vacuum and can be evaluated and transformed into insight only if it is contextualized with associated activities. This is best accomplished in collaboration with the Institutional Research (IR) office. The Association for Institutional Research’s aspirational statement “highlights the need for IR to serve a broader range of decision makers.”

We emphasize the critical need to incorporate fundamental knowledge of experimental and quasi-experimental design at the beginning of any grant project. In essence, grant projects are experiments—just not necessarily being performed in a laboratory. The design of any experiment is to introduce new conditions. The independent variable is the grant project and the dependent variable is the success of the target population (students, faculty). The ability to properly measure and replicate this scientific process must be established during project planning, and the IR office can be instrumental in the design of your evaluation.

Responding to a program solicitation (or RFP, RFA, etc.) provides the opportunity to establish the need for the project, measurable outcomes, and an appropriate plan for evaluation that can win over the hearts and minds of reviewers, and lead to a successful grant award. Institutional researchers work with the grant office not only to measure outcomes but also to investigate and provide potential opportunities for improvement. IR staff act as data scientists and statisticians while working with grants and become intimately acquainted with the data, collection process, relationships between variables, and the science being investigated. While the term statistician and data scientist are often used synonymously, data scientists do more than just answer hypothesis tests and develop forecasting models; they also identify how variables not being studied may affect outcomes. This allows IR staff to see beyond the questions that are being asked and not only contribute to the development of the results but also identify unexpected structures in the data. Finding alternative structure may lead to further investigation in other areas and more opportunities for other grants.

If a project’s objective is to affect positive change in student retention, it is necessary to know the starting point before any grant-funded interventions are introduced. IR can provide descriptive statistics on the student body and target population before the intervention. This historical data is used not only for trend analysis but also for validation, correcting errors in the data. Validation can be as simple as looking for differences between comparison groups and confirming potential differences are not due to error. IR can also assist with the predictive analytics necessary to establish appropriate benchmarks for measurable objectives. For example, predicting that an intervention will increase retention rates by 10-20% when a 1-2% increase would be more realistic could lead to a proposal being rejected or set the project up for failure. Your IR office can also help ensure that the appropriate quantitative statistical methods are used to analyze the data.

Tip: Involve your IR office from the beginning, during project planning. This will contribute greatly to submitting a competitive application, the evaluation of which provides the guidance necessary for a successful project.