Marilyn Barger

Executive Director, FLATE – Florida Advanced Technological Education Center of Excellence

Dr. Marilyn Barger is the Executive Director and Principle Investigator for the Florida Advanced Technological Education Center of Excellence (FL-ATE). This center focuses on high-technology manufacturing. Dr. Barger is also on EvaluATE’s Community College Liason Panel.


Blog: This is How “We” Do Data: Collect, Unravel, Summarize!

Posted on November 19, 2014 by  in Blog ()

Executive Director, FLATE – Florida Advanced Technological Education Center of Excellence

Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

At this time of year (early spring), we at FLATE stop to review, tally, analyze, and aggregate our collected data from the previous calendar year. We look for new comparative data sets from other organizations to see how we are doing in specific activities, or groups of activities related to FLATE’s outreach, curriculum reform and professional development efforts. We look for ways to refine our data collection by identifying unclear survey questions or gaps in information we retrieve from various surveys. Here’s how we do it and how it all comes together:

We start with a good review and scrub off raw data that has been collected and recorded. Although we regularly review data and trends during the year to prepare for our year-end summaries, we take a final look to be sure all the data we use is “good data.” Once this somewhat tedious stage is complete, the treasure hunt for “hidden data nuggets” begins.

Many times we are just looking for summary data that we have to report and trends, hopefully positive trends, in the impact of our activities, resources, and events. Beyond just reporting, this kind of information tells us if we are still reaching the same kinds of participants and the same numbers of stakeholders for different kinds of events, resources, and activities. Reviewing this information carefully helps us target specific “missed” audiences in the coming year at various events.

After we complete reviewing, cleaning, organizing, and summarizing our annual reporting data, we continue to probe and assess what else we can learn from it. In various stages of data review and summarizing, we often find ourselves asking: “I wonder if…?”; “how could I know if this is connected to that?” or “wouldn’t it be great if we also could know…?” These are questions we revisit by bringing data together from different sources and looking at the data from different perspectives. The exercise becomes a game of puzzling together different results, trying to reveal more impact.

We move from the observations “wows,” “oh my’s,” and “ah-ha’s!” to filtering which of the ideas or questions will give us the most “bang for our buck,” as well as help us better answer the questions that NSF and our valued stakeholders ask. The cycle of continuous improvement underlies this whole process. How can we do what we do better by being more strategic in our activities, events, and resources? Can we ask better survey questions that will reveal more and better information? Can we totally change our approach to get more impactful data?

Here is an example using our websites and blogs data: FLATE collects monthly data using Google Analytics for its websites and newsletter and reviews this information quarterly with staff and the leadership team. The objective is to compare website and newsletter activity and growth (as measured by visits) against benchmarks and evaluate usage trends. The process of collecting and taking a look at this information led to a further question and action item: In addition to increasing use of these FLATE products, we now wish to increase the percentage of returning visitors, providing evidence that our community of practice not only uses, but relies on FLATE as an information resource.

Newsletter: Meet EvaluATE’s Community College Liaison Panel

Posted on January 1, 2014 by , , , in Newsletter - ()

The ATE program is community college-based, and as such EvaluATE places a priority on meeting the needs of this constituency. To help ensure the relevancy and utility of its resources, EvaluATE has convened a Community College Liaison Panel (CCLP). CCLP members Michael Lesiecki, Marilyn Barger, Jane Ostrander, and Gordon Snyder are tasked with keeping the EvaluATE team tuned into the needs and concerns of 2-year college stakeholders and engaging the ATE community in the review and pilot testing of EvaluATE-produced materials.

These resources distill relevant elements of evaluation theory, principles, and best practices so that a user can quickly understand and apply them for a specific evaluation-related task. They are intended to support members of the ATE community to enhance the quality of their evaluations.

The CCLP’s role is to coordinate a three-phase review process. CCLP members conduct a first-level review of an EvaluATE resource. The EvaluATE team revises it based on the CCLP’s feedback, then each of the four CCLP members reaches out to diverse members of the ATE community—PIs, grant developers, evaluators, and others—to review the material and provide confidential, structured feedback and suggestions. After another round of revisions, the CCLP engages another set of ATE stakeholders to actually try out the resource to ensure it “works” as intended in the real world. Following this pilot testing, EvaluATE finalizes the resource for wide dissemination.

The CCLP has shepherded two resources through the entire review process: the ATE Evaluation Primer and ATE Evaluation Planning Checklist. In the hopper for review in the next few months are the ATE Logic Model Template and Evaluation Planning Matrix, Evaluation Questions Checklist, ATE Evaluation Reporting Checklist, and Professional Development Feedback Survey Template. In addition, CCLP members are leading the development of a Guide to ATE Evaluation Management—by PIs for PIs.

The CCLP invites anyone interested in ATE evaluation to participate in the review process. For a few hours of your time, you’ll get a first look at and tryout of new resources. And your inputs will help shape and strengthen the ATE evaluation community. We also welcome recommendations of tools and materials that others have developed that would be of interest to the ATE community.

To get involved, email CCLP Director Mike Lesiecki at mlesiecki@gmail.com. Tell him you would like to help make EvaluATE be the go-to evaluation resource for people like yourself.