Archive: culture

Blog: Attending to culture, diversity, and equity in STEM program evaluation (Part 2)

Posted on May 9, 2018 by  in Blog ()

Assistant Professor, Department of Educational Research Methodology, University of North Carolina Greensboro

Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

In my previous post, I gave an overview of two strategies you can use to inform yourself about the theoretical aspect of engagement with culture, diversity, and equity in evaluation. I now present two practical strategies, which I believe should follow the theoretical strategies presented in my previous post.

Strategy three: Engage with related sensitive topics informally

To begin to feel comfortable with these topics, engage with these issues during interactions with your evaluation team members, clients, or other stakeholders. Evaluators should acknowledge differing stakeholder opinions, while also attempting to assist stakeholders in surfacing their own values, prejudices, and subjectivities (Greene, Boyce, & Ahn, 2011).

To do this, bring up issues of race, power, inequity, diversity, and culture for dialogue in meetings, emails, and conversations (Boyce, 2017). Call out and discuss micro-aggressions (Sue, 2010) and practice acts of micro-validation (Packard, Gagnon, LaBelle, Jeffers, & Lynn, 2011). For example, when meeting with clients, you might ask them to discuss how they plan to ensure not just diversity but inclusivity within their program. You also can ask them to chart out program goals through a logic model but also ask them to consider if they think underrepresented participants might experience the program differently than their majority participants. Ask clients if they have considered cultural sensitivity training for program managers and/or participants.

Strategy four: Attend to issues of culture, equity, and diversity formally

Numerous scholars have addressed the implications of cultural responsiveness in practice (Frierson, Hood, Hughes, & Thomas, 2010; Hood, Hopson, & Kirkhart, 2015), with some encouraging contemplation surrounding threats to, as well as evidence for, multicultural validity by examining relational, consequential, theoretical, experiential, and methodological justificatory perspectives (Kirkhart, 2005, 2010). I believe the ultimate goal is to be able to attend to culture and context in all formal aspects of the research and evaluation. It is especially important to take a strengths-based, anti-defect approach (Chun & Evans, 2009) and focus on research intersectionality (Collins, 2000).

To do this, you can begin with the framing of the program goals. My programs aim to give underrepresented minorities in STEM skills to survive in the field. This perspective assumes that something is inherently wrong with these students. Instead, think about rewording evaluation questions to examine the culture of the department or program, to explore why more underrepresented groups (at least to have parity with the percentage in population) don’t thrive. Further, evaluators can attempt to include these topics in evaluation questions, develop culturally commensurate data instruments, and be sensitive to these issues during data collection, analysis, and reporting. Challenge yourself to think about this attendance as more than the inclusion of symbolic and politically correct buzzwords (Boyce & Chouinard, 2017), but as a true infusion of these aspects into your practice. For example, I always include an evaluation question about diversity, equity, and culture in my evaluation plans.

These two blog posts are really just the tip of the iceberg. I hope you find these strategies useful as you begin to engage with culture, equity, and diversity in your work. As I previously noted, I have included citations throughout so that you can read more about these important concepts. In a recently published article, my colleague Jill Anne Chouinard and I discuss how we trained evaluators to work through these strategies in a Culturally Responsive Approaches to Research and Evaluation course (Boyce & Chouinard, 2017).


Blog: Attending to culture, diversity, and equity in STEM program evaluation (Part 1)

Posted on May 1, 2018 by  in Blog ()

Assistant Professor, Department of Educational Research Methodology, University of North Carolina Greensboro

Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

The conversation, both practical and theoretical, surrounding culture, diversity, and equity in evaluation has increased in recent years. As many STEM education programs aim to broaden participation of women, ethnic minority groups, and persons with disabilities, attention to culture, diversity, and equity is paramount. In two blog posts, I will provide a brief overview of four strategies to meaningfully and respectfully engage with these important topics. In this first blog, I will focus on strategies that are helpful in learning more about these issues but that are theoretical and not directly related to evaluation practice. I will also help you learn more about these issues. I should note that I purposely have included a number of citations so that you may read further about these topics.

Strategy one: Recognize social inquiry is a cultural product

Social science knowledge of minority populations, constructed with narrow worldviews, has demeaned characteristics, distorted interpretations of conditions and potential, and remained limited in its capacity to inform efforts to improve the life chances of historically disadvantaged populations (Ladson-Billings, 2000). Begin by educating yourself about the role communicentric bias—the tendency to make one’s own community, often the majority class, the center of conceptual frames that constrains all thought (Gordon, Miller, & Rollock, 1990)—and individual, institutional, societal, and civilizational racism play in education and the social sciences (Scheurich & Young, 2002). Seek to understand the culture, context, historical perspectives, power, oppressions, and privilege in each new context (Greene, 2005; Pon, 2009).

To do this, you can read and discuss books, articles, and chapters related to epistemologies— theories of knowledge—of difference, racialized discourses, and critiques about the nature of social inquiry. Some excellent examples include Stamped from the Beginning by Ibram X. Kendi, The Shape of the River by William G. Bowen and Derek Bok, and Race Matters by Cornel West. Each of these books is illuminating and a must-read as you begin or continue your journey to better understand race and privilege in America. Perhaps start a book club so that you can process these ideas with colleagues and friends.

Strategy two: Locate your own values, prejudices, and identities

The lens through which we view the world influences all evaluation processes, from design to implementation and interpretations (Milner, 2007; Symonette, 2015). In order to think crtically bout issues of culture, power, equity, class, race, and diversity, evaluators should understand their own personal and cultural values (Symonette, 2004). As Peshkin (1988) has noted, the practice of locating oneself can result in a better understanding of one’s own subjectivities. In my own work, I always attempt to acknowledge the role my education, gender, class, and ethnicity will play in my work.

To do this, you can reflect on your own educational background, personal identities, experiences, values, prejudices, predispositions, beliefs, and intuition. Focus on your own social identity, the identities of others, whether you belong to any groups with power and privilege, and how your educational background and identities shape your beliefs, role as an evaluator, and experiences. To unearth some of the more underlying values, you might consider participating in a privilege walk exercise and reflecting on your responses to current events.

These two strategies are just the beginning. In my second blog post, I will focus on engaging with these topics informally and formally within your evaluation practice.


Newsletter: Developing a Culture of Evaluation

Posted on October 1, 2013 by  in Newsletter - ()

Principal Research Scientist, Education Development Center, Inc.

As an ATE project, you and your team collect a lot of data: You complete the annual monitoring survey, you work with your evaluator to measure outcomes, you may even track your participants longitudinally in order to learn how they integrate their experiences into their lives. As overwhelming as it may seem at times to manage all the data collection logistics and report writing, these data are important to telling the story of your project and the ATE program. Developing a culture of evaluation in your project and your organization can help to meaningfully put these data to use.

Fostering a culture of evaluation in your project means that evaluation practices are not disconnected from program planning, implementation, and reporting. You’re thinking of evaluation in planning project activities and looking for ways to use data to reflect on and improve your work. During implementation, you consult your evaluator regularly so that you can hear what they’re learning from the data collection, and ensure that they know what’s new in the project. And at analysis and reporting times, you’re ensuring that the right people are thinking about how to use the evaluation findings to make improvements and demonstrate your project’s value to important stakeholder audiences. You and your team are reflecting on how the evaluation went and what can be improved. In a project that has an “evaluation culture,” evaluators are partners, collecting important information to inform decision making.

A great example of evaluators-as partners came from an NSF PI who shared that he regularly talks with his evaluator, peer-to-peer, about the state of the field, not just about his particular project. He wants to now what his evaluator is learning about practice in the field from other projects, workshops, conferences and meetings, and he uses these insights to help him reflect on his own work.