Melanie Hwalek

CEO, SPEC Associates

Melanie Hwalek is CEO of SPEC Associates, a nonprofit program evaluation organization headquartered in downtown Detroit with both a local and national scope to services. Melanie is a veteran at program evaluation. In her long career, she has evaluated more than 200 programs for more than 150 different organizations. Melanie teaches evaluation management for Michigan State University’s M.A. in program evaluation. She was a board member of both the American Evaluation Association and the Michigan Association for Evaluation. She holds the Certified Evaluator Designation from the Canadian Evaluation Society (there is no equivalent national certification in the U.S.)


Blog: Evaluation Management Skill Set

Posted on April 12, 2017 by  in Blog ()

CEO, SPEC Associates

Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

We as evaluators, all know that managing an evaluation is quite different from managing a scientific research project. Sure, we need to take due diligence in completing the basic inquiry tasks:  deciding study questions/hypotheses; figuring out the strongest design, sampling plan, data collection methods and analysis strategies; and interpreting/reporting results. But evaluation’s purposes extend well beyond proving or disproving a research hypothesis. Evaluators must also focus on how the evaluation will lead to enlightenment and what role it plays in support of decision making. Evaluations can leave in place important processes that extend beyond the study itself, like data collection systems and changed organizational culture that places greater emphasis on data-informed decision making. Evaluations also exist within local and organizational political contexts, which are of less importance to academic and scientific research.

Very little has been written in the evaluation literature about evaluation management. Compton and Baizerman are the most prolific authors editing two issues of New Directions in Evaluation on the topic. They approach evaluation management from a theoretical perspective, discussing issues like the basic competencies of evaluation managers within different organizational contexts (2009) and the role of evaluation managers in advice giving (2012).

I would like to describe good evaluation management in terms of the actual tasks that an evaluation manager must excel in—what evaluation managers must be able to actually do. For this, I looked to the field of project management. There is a large body of literature about project management, and whole organizations, like the Project Management Institute, dedicated to the topic. Overlaying evaluation management onto the core skills of a project manager, here is the skill set I see as needed to effectively manage an evaluation:

Technical Skills:

  • Writing an evaluation plan (including but not limited to descriptions of basic inquiry tasks)
  • Creating evaluation timelines
  • Writing contracts between the evaluation manager and various members of the evaluation team (if they are subcontractors), and with the client organization
  • Completing the application for human subjects institutional review board (HSIRB) approval, if needed

Financial Skills:

  • Creating evaluation budgets, including accurately estimating hours each person will need to devote to each task
  • Generating or justifying billing rates of each member of the evaluation team
  • Tracking expenditures to assure that the evaluation is completed within the agreed-upon budget

Interpersonal Skills:

  • Preparing a communications plan outlining who needs to be apprised of what information or involved in which decisions, how often and by what method
  • Using appropriate verbal and nonverbal communication skills to assure that the evaluation not only gets done, but good relationships are maintained throughout
  • Assuming leadership in guiding the evaluation to its completion
  • Resolving the enormous number of conflicts that can arise both within the evaluation team and between the evaluators and the stakeholders

I think that this framing can provide practical guidance for what new evaluators need to know to effectively manage an evaluation and guidance for how veteran evaluators can organize their knowledge for practical sharing. I’d be interested in comments as to the comprehensiveness and appropriateness of this list…am I missing something?

Blog: Maximizing Stakeholder Engagement by Bringing Evaluation to Life!

Posted on April 13, 2016 by  in Blog ()

CEO, SPEC Associates

Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Melanie is CEO of SPEC Associates, a nonprofit program evaluation and process improvement organization headquartered in downtown Detroit.  Melanie is also on the faculty of Michigan State University, where she teaches Evaluation Management in the M.A. in Program Evaluation program. Melanie holds a Ph.D. in Applied Social Psychology and has directed evaluations for almost 40 years both locally and nationally. Her professional passion is making evaluation an engaging, fun and learning experience for both program stakeholders and evaluators. To this end, Melanie co-created EvaluationLive!, an evaluation practice model that guides evaluators in ways to breathe life into the evaluation experience.

Why is it that sometimes meetings with evaluation stakeholders seem to generate anxiety and boredom, while other times they generate excitement, a hunger for learning and, yes, even fun!?

My colleague, Mary Williams, and I started wondering and defining this about eight years ago. With 60 years of collective evaluation experience, we documented, analyzed cases, and conducted an in-depth literature review seeking an answer. We honed in on two things: (1) a definition of what exemplary stakeholder engagement looks and feels like, and (2) a set of factors that seem to predict when maximum stakeholder engagement exists.

To define “exemplary stakeholder engagement” we looked to the field of positive psychology and specifically to Mihaly Csikszentmihalyi’s  (2008) Flow Theory. Csikszentmihalyi defines “flow” as that highly focused mental state where time seems to stand still. Think of a musician composing a sonata. Think of a basketball player being in the “zone.” Flow theory says that this feeling of “flow” occurs when the person perceives that the task at hand is challenging and also perceives that she or he has the skill level sufficient to accomplish the task.

The EvaluationLive! model asserts that maximizing stakeholder engagement with an evaluation – having a flow-like experience during encounters between the evaluator and the stakeholders – requires certain characteristics of the evaluator/evaluation team, of the client organization, and of the relationship between them. Specifically, the evaluator/evaluation team must (1) be competent in the conduct of program evaluation; (2) have expertise in the subject matter of the evaluation; (3) have skills in the art of interpersonal, nonverbal, verbal and written communication; (4) be willing to be flexible in order to meet stakeholders’ needs typically for delivering results in time for decision making; and (5) approach the work with a non-egotistical learner attitude. The client organization must (1) be a learning organization open to hearing good, bad, and ugly news; (2) drive the questions that the evaluation will address; and (3) have a champion positioned within the organization who knows what information the organization needs when, and can put the right information in front of the right people at the right time. The relationship between the evaluator and client must be based on (1) trust, (2) a belief that both parties are equally expert in their own arenas, and (3) a sense that the evaluation will require shared responsibility on the part of the evaluator and the client organization.

Feedback from the field shows EvaluationLive!’s goalposts help evaluators develop strategies to emotionally engage clients in their evaluations. EvaluationLive! has been used to diagnose problem situations and to direct “next steps.” Evaluators are also using the model to guide how to develop new client relationships. We invite you to learn and get involved.

EvaluationLive! Model

Summary of the EvaluationLive! Model