Megan Mullins

President, Mullins Consulting, Inc.

Megan Mullins is a program evaluator located in Muncie, Indiana. She specializes in evaluation and social science research supporting education, nonprofit development, and community change initiatives. She emphasizes communication, collaboration, shared experiences, and shared ownership of initiatives in which she is engaged. Mullins works with diverse people from a wide variety of community sectors on issues about which they care deeply. She uses an appreciative approach to project development and implementation, and uses program evaluation to illuminate process and improve impact in ways that help her partners tell dynamic, scientifically valid, and relevant stories about their work.


Blog: Building ATE Social Capital Through Evaluation Activities

Posted on February 24, 2021 by  in Blog () ()

President, Mullins Consulting, Inc.

Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

“Social networks have value. Social capital refers to the value of social networks, or whom people know, and the inclinations that arise from these networks to do things for each other. Thus, people benefit from the trust, reciprocity, information, and cooperation of these social networks” (Robert D. Putnam, Harvard Kennedy School of Government, 2018).

Within the context of “new-to-ATE” grants, many novice PIs have low social capital compared to more experienced PIs. New PIs are often not familiar with the norms of NSF grant proposal writing, reporting, and other communication; other PIs and collaborators in the community; and other elements that empower more experienced PIs. While proposal-writing mentoring programs are available, not all ATE applicants are granted this opportunity, and this mentoring typically ends once a program is funded.

The evaluator is in a unique position to strengthen social capital by offering new PIs access to their client pool of ATE grantees to facilitate networking and the sharing of information. Connections can be made through the evaluator, new knowledge shared, and relationships cultivated. Increasing access to networks and information can lead to stronger program implementation strategies as well as increased PI confidence in the process.

Here are three tips on when and how an evaluator can connect clients to each other.

1.     The First Six Months. My evaluation team continually discusses how the ATE programs we are evaluating might logically connect (e.g., discipline/area, program components). When a challenge arises, we see what connections can be made so that novice PIs have someone to use as a resource in navigating the challenge. Most experienced PIs are willing to share their experiences in order help others.

2.     National ATE PI Conference. As a lead evaluator, I find time at the ATE conference to introduce clients to one another over coffee or before or after sessions being attended by my clients. I preface these face-to-face meetings with inquiries beforehand to make sure clients are interested in meeting and have available time. Most report it helpful to meet others in similar fields and get a chance to talk to each other about their programs.

3.     Year One Reporting Time. I have found that, traditionally, novice ATE PIs are very anxious about writing their first annual report to the NSF. To address this challenge, I established a meeting of new and more experienced PIs to discuss year one reporting. In the meeting, a seasoned PI presents how they approached first-year reporting and answers questions alongside a former NSF program officer who provides further guidance. The positive feedback from this meeting has been tremendous.

Connecting new PIs with more experienced PIs facilitates the growth of social capital, resulting in better collaborative inquiry, stronger networks, persistence with project implementation, and subsequent reporting of impact.

 

Reference:

Harvard Kennedy School of Government. (2018). ​Social Capital Primer. http://robertdputnam.com/bowling-alone/social-capital-primer/

 

Blog: Successful Practices in ATE Evaluation Planning

Posted on July 19, 2018 by  in Blog ()

President, Mullins Consulting, Inc.

Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

In this essay, I identify what helps me create a strong evaluation plan when working with new Advanced Technological Education (ATE) program partners. I hope my notes add value to current and future proposal-writing conversations.

Become involved as early as possible in the proposal-planning process. With ATE projects, as with most evaluation projects, the sooner an evaluator is included in the project planning, the better. Even if the evaluator just observes the initial planning meetings, their involvement helps them become familiar with the project’s framework, the community partnerships, and the way in which project objectives are taking shape. Such involvement also helps familiarize the evaluator with the language used to frame project components and the new or established relationships expected for project implementation.

Get to know your existing and anticipated partners. Establishing or strengthening partnerships is a core component of ATE planning, as ATE projects often engage with multiple institutions through the creation of new certifications, development of new industry partnerships, and explanation of outreach efforts in public schools. The evaluator should take detailed notes on the internal and external partnerships involved with the project. Sometimes, to support my own understanding as an evaluator, it helps for me to visually map these relationships. Also, the evaluator should prepare for the unexpected. Sometimes, partners will change during the planning process as partner roles and program purposes become more clearly defined.

Integrate evaluation thinking into conversations early on. Once the team gets through the first couple of proposal drafts, it helps if the evaluator creates an evaluation plan and the team makes time to review it as a group. This will help the planning team clarify the evaluation questions to be addressed and outcomes to be measured. This review also allows the team to see how their outcomes can be clearly attached to program activities and measured through specific methods of data collection. Sometimes during this process, I speak up if a component could use further discussion (e.g., cohort size, mentoring practices). If an evaluator has been engaged from the beginning and has gotten to know the partners, they have likely built the trust necessary to add value to the discussion of the proposal’s central components.

Operate as an illuminator. A colleague I admire once suggested that evaluation be used as a flashlight, not as a hammer. This perspective of prioritizing exploration and illumination over determination of cause and effect has informed my work. Useful evaluations certainly require sound evaluation methodology, but they also require the crafting of results into compelling stories, told with data guiding the way. This requires working with others as interpretations unfold, discovering how findings can be communicated to different audiences, and listening to what stakeholders need to move their initiatives forward.

ATE programs offer participants critical opportunities to be a part of our country’s future workforce. Stakeholders are passionate about their programs. Careful, thoughtful engagement throughout the proposal-writing process builds trust while contributing to a quality proposal with a strong evaluation plan.