Like many of you, I wear multiple professional hats. Critical thinking skills are at the nexus of all my roles. The importance of improving critical thinking transcends disciplines, even though the contexts and applications vary. As a sociologist, I see the how the deficit of critical thinking skills has a negative impact on society. As an evaluator, I find that these skills are frequent targets for NSF projects across disciplines.

Identifying important skills and implementing strategies to improve them is only one part of a grant proposal. An equally challenging issue is finding appropriate assessments.

Through the years I have learned some useful tips in selecting an instrument to best complement your evaluation needs. You should select an assessment that:

    1. Aligns with the skills that are important to your project.
    2. Is transparent about both the questions and the rubric for assessing those questions.
    3. Provides insight and/or training as to how you can improve the skills.
    4. Provides flexible comparison groups for you (e.g., pre and post, or national user norms).
    5. Provides reports that are easy to read and use.
    6. Demonstrates validity, reliability, and cultural fairness.

As we struggled to find assessment options that could meet these standards ourselves, we developed and refined the Critical-thinking Assessment Test (CAT). If you are seeking to improve students’ critical thinking skills, you may want to consider this instrument.

The Critical-thinking Assessment Test (CAT)

This NSF-funded instrument is the product of 20 years’ extensive development, testing, and refinement with faculty and students from over 350 institutions and over 40 NSF projects. One innovation of this assessment is its integration of short-answer essay questions based on real-world situations. It provides quantitative and qualitative data on the skills that faculty believe are most important for their students to have 10 years after graduating.

Skills Assessed by the CAT:

Evaluating Information

    • Separate factual information from inferences.
    • Interpret numerical relationships in graphs.
    • Understand the limitations of correlational data.
    • Evaluate evidence and identify inappropriate conclusions.

Creative Thinking

    • Identify alternative interpretations for data and observations.
    • Identify new information that might support or contradict a hypothesis.
    • Explain how new information can change a problem.

Learning and Problem Solving

    • Separate relevant from irrelevant information.
    • Integrate information to solve problems.
    • Learn and apply new information.
    • Use mathematical skills to solve real-world problems.

Communication

    • Communicate ideas effectively.

Our team truly enjoys working with evaluators and PIs to help them assess these skills and provide evidence of their success. Some NSF projects and courses have made gains in critical thinking equivalent to those gained in an entire four-year college experience. Our partner institutions have experienced positive outcomes, growth, and learning from working with the CAT.

You can find more information about the CAT here, or you can contact me with any questions you have.

 

References:

Haynes, A., Lisic, E., Goltz, M., Stein, B., & Harris, K. (2016). Moving beyond assessment to improving students’ critical thinking skills: A model for implementing change. Journal of the Scholarship of Teaching and Learning16(4), 44–61.

Stein, B., & Haynes, A. (2011). Engaging faculty in the assessment and improvement of students’ critical thinking using the CAT. Change: The Magazine of Higher Learning, 43, 44–49.

About the Authors

Ada Haynes

Ada Haynes box with arrow

Professor of Sociology and Co-Director of the Center for Assessment and Improvement of Learning

Ada Haynes is a professor of sociology and co-director of the Center for Assessment and Improvement of Learning (CAIL) at Tennessee Technological University. She is also an independent evaluator on NSF STEM grants focusing on improving student learning and faculty development from multiple NSF divisions, including ATE. Over the past 20 years she has worked on assessing and improving critical thinking, including serving as the co-PI on three NSF grants to develop, refine, and disseminate the Critical-thinking Assessment Test (CAT) and to establish the CAIL center.

Creative Commons

Except where noted, all content on this website is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Related Blog Posts

Nation Science Foundation Logo EvaluATE is supported by the National Science Foundation under grant number 1841783. Any opinions, findings, and conclusions or recommendations expressed on this site are those of the authors and do not necessarily reflect the views of the National Science Foundation.