Humber College, as part of a learning outcomes assessment consortium funded by the Higher Education Quality Council of Ontario (HEQCO), has developed an assessment tool to measure student gains in critical thinking (CT) as expressed through written communication (WC).
In Phase 1 of this project, a cross-disciplinary team of faculty and staff researched and developed a tool to assess students’ CT skills through written coursework. The tool was tested for usability by a variety of faculty and in a variety of learning contexts. Based on this pilot, we revised the tool to focus on two CT dimensions: comprehension and integration of writer’s ideas, within which are six variables: interpretation, analysis, evaluation, inference, explanation, and self-regulation.
In Phase 2, our key questions were:
- What is the validity and reliability of the assessment tool?
- Where do students experience greater levels of CT skill achievement?
- Are students making gains in learning CT skills over time?
- What is the usability and scalability of the tool?
To answer the first question, we examined the inter-rater reliability of the tool, as well as compared CTWC assessment scores with students’ final grades. We conducted a cross-sectional analysis by comparing diverse CT and WC learning experiences in different contexts, namely our mandatory semester I and II cross-college writing courses, where CTWC skills are taught explicitly and reinforced as course learning outcomes; vocationally-oriented courses in police foundations, where the skills are implicitly embedded as deemed essential by industry; and a critical thinking course in our general arts and sciences programs, where CT is taught as content knowledge.
We also performed a longitudinal analysis by assessing CTWC gains in a cohort of students across two semesters in their mandatory writing courses.
Overall, our tests showed positive results for reliability and validity. Our cross-sectional analysis showed the greatest CT gains in courses where the skill is explicitly taught. Our longitudinal analysis showed only modest gains, indicating that a two-semester span is insufficient for significant improvement to occur.
In terms of usability, faculty agreed that the revised tool was straightforward and easy to apply. However, there was less agreement on the tool’s meaningfulness to students, indicating that further research needs to include student feedback.
Lessons learned:
- Build faculty buy-in at the outset and recognize workload issues
- Ensure project team members are qualified
- For scalability, align project with other institutional priorities
Recommendations:
- Teach CT explicitly and consistently, as a skill, and over time
- Strategically position courses where CT is taught explicitly throughout a program for maximum reinforcement
- Assess and provide feedback on students’ skills at regular intervals
- Implement faculty training to build a common understanding of the importance of essential skills and their assessment
- For the tool to be meaningful, students must understand which skills are being assessed and why
Our project will inform Humber’s new Essential Skills Strategy, which includes the development of an institutional learning outcomes framework and assessment process.
A detailed report, including our assessment tool, will be available through HEQCO in the near future. For further information, please contact the authors: vera.beletzan@humber.ca or paula.gouveia@humber.ca
Vera Beletzan 
Senior Special Advisor Essential Skills Humber College
Vera Beletzan is Senior Special Advisor Essential Skills at Humber College, Canada. She is an experienced academic and administrative leader in the Ontario college system. Her background includes teaching, curriculum development, teacher training, and faculty professional development. She has held a number of positions at Humber College, most recently as Associate Dean in the Department of English. She has worked extensively in the areas of student access, success, and engagement. She served a leadership role in the delivery of programming and services to support writing, critical thinking and English for speakers of other languages (ESOL) skill development. She leads Humber’s essential skills strategy, including the development of the college’s first institutional learning outcomes framework, and is a lead on Humber’s team in the Higher Education Quality Council of Ontario’s Learning Outcomes Assessment Consortium.
Paula Gouveia
Dean, School of Liberal Arts and Sciences Humber College
Dr. Paula Gouveia is Dean at the School of Liberal Arts and Sciences at Humber College, Canada. She has extensive experience in higher education academic and administrative leadership, curriculum design and delivery, and teaching. As Dean of the School of Liberal Arts and Sciences, she is responsible for the development and delivery of cross-institutional English, mathematics, and liberal studies programming. Paula also leads the development and delivery of innovative programs and services that support student success and engagement, such as transfer programs at the post-secondary and pre- and post-secondary levels, including Humber's General Arts and Science Programs, English Language Centre and Department of Academic Upgrading, as well as the college Math and Writing Centres. Paula is an institutional lead on the Higher Education Quality Council of Ontario's Learning Outcomes Assessment Consortium and a provincial lead on the Ontario College Math Test.
Except where noted, all content on this website is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
EvaluATE is supported by the National Science Foundation under grant number 1841783. Any opinions, findings, and conclusions or recommendations expressed on this site are those of the authors and do not necessarily reflect the views of the National Science Foundation.