Archive: data quality

Blog: Tips for Building and Strengthening Stakeholder Relationships

Posted on November 23, 2020 by  in Blog ()

Project Manager, EvaluATE at The Evaluation Center

Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Hello! I am Valerie Marshall, I work on a range of projects at The Evaluation Center, including EvaluATE, where I serve as the administrator and analyst for the annual ATE Survey.

A cornerstone of evaluation is working with stakeholders. Stakeholders are individuals or groups who are part of an evaluation or are otherwise interested in its findings. They may be internal or external to the program being evaluated.

Stakeholders’ interests and involvement in evaluation activities may vary. But they are a key ingredient to evaluation success. They can provide critical insight into project activities and evaluation questions, serve as the gatekeepers to other stakeholders or data, and help determine if evaluation findings and recommendations are implemented.

Given their importance, identifying ways to build and nurture relationships with stakeholders is pivotal.

So the question is: how can you build relationships with evaluation stakeholders?

Below is a list of tips based on my own research and evaluation experience. This list is by no means exhaustive. If you are an ATE PI or evaluator, please join EvaluATE’s Slack community to continue the conversation and share some of your own tips!

Tip 1: Be intentional and adaptative about how you communicate. Not all stakeholders will prefer the same mode of communication. And how stakeholders want to communicate can change over the course a project’s lifecycle. In my experience, using communication styles and tools that align with stakeholders’ needs and preferences often results in greater engagement. So, ask stakeholders how they would like to communicate at various points throughout your work together.

Tip 2: Build rapport. ATE evaluator and fellow blogger George Chitiyo previously noted that building rapport with stakeholders can make them feel valued and, in turn, help lead to quality data. Rapport is defined as a friendly relationship that makes communication easier (Merriam-Webster). Chatting during “down time” in a videoconference, sharing helpful resources, and mentioning a lighthearted story are great ways to begin fostering a friendly relationship.

Tip 3: Support and maintain transparency. Communicate with stakeholders about what is being done, when, and why. This not only reduces confusion but also facilitates trust. Trust is pivotal to building  productive, healthy relationships with stakeholders. Providing project staff with a timeline of research or evaluation activities, giving regular progress updates, and meeting with stakeholders one-on-one or in small groups to answer questions or address concerns are all helpful ways to generate transparency.

Tip 4: Identify roles and responsibilities. When stakeholders know what is expected of them and how they can and cannot contribute to different aspects of a research or evaluation project, they can engage in a more meaningful way. The clarity generated from the process of outlining the roles and responsibilities of both stakeholders and research and evaluation staff can help reduce misunderstandings. At the beginning of a project, and as new staff and stakeholders join the project, make sure to review roles and expectations with everyone.

Blog: Improving the Quality of Evaluation Data from Participants

Posted on June 10, 2020 by  in Blog ()

Professor of Educational Research and Evaluation, Tennessee Tech University

Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Improving the Quality of Evaluation Data from Participants

I have had experience evaluating a number of ATE projects, all of them collaborative projects among several four-year and two-year community colleges. One of the projects’ overarching goals is to provide training to college instructors as well as elementary-, middle-, and high-school teachers, to advance their skills in additive manufacturing and/or smart manufacturing.

The training is done via the use of train-the-trainer studios (TTS). TTSs provide novel hands-on learning experiences to program participants. As with any program, the evaluation of such projects needs to be informed by rich data to capture participants’ entire experience, including the knowledge they gain.

Here’s one lesson I’ve learned from evaluating these projects: Participants’ perception of their value in the project contributes crucially to the quality of data collected.

As the evaluator, one can make participants feel that the data they are being asked to provide (regarding technical knowledge gained, their application of it, and perceptions about all aspects of the training) will be beneficial to the overall program and to them directly or indirectly.

If they feel that their importance is minimal, and that the information they provide will not matter, they will provide the barest amount of information (regardless of the method of data collection employed). If they understand the importance of their participation, they’re more likely to provide rich data.

How can you make them feel valued?

Establish good rapport with each of the participants, particularly if the group(s) is(are) of reasonable size. Make sure to interact informally with each participant throughout the training workshop(s). Inquire about their professional work, and ask them about supports that they might need when they return to their workplace.

The responses to the open-ended questions on most of my workshop evaluations have been very rich and detailed¾much more so than those from participants to whom I administered the survey remotely, without ever meeting. Program participants want to connect to a real person, not a remote evaluator. In the event that in-person connections are not possible, explore other innovative ways of establishing rapport with individual participants, before and during the program.

How can you improve the quality of data they will provide?

 Sell the evaluation. Make it clear how the evaluation findings will be used and how the results will benefit the participants and their constituents specifically, directly or indirectly.

 Share success stories. During the training workshops that I have been evaluating, I’ve shared some previous success stories with participants in order to show them what they are capable of accomplishing as well.

The time and energy you spend building these connections with participants will result in high-quality evaluation data, ultimately helping the program serve participants better.