By: Jeff Cain, EdD, MS
In 2001, despite amassing 23 total bases, the San Francisco Giants lost 1-0 to the Chicago Cubs. Baseball scores are not determined by the number of singles, doubles, walks, etc. that a team collects throughout the game. The only standard that matters is whether a runner successfully crosses home plate.
This baseball metaphor applies to the specifications grading schema I adopted for my Pharmacy Operations & Financial Management course in Spring 2023. Specifications grading eliminates the collection of points from exams and assignments in favor of a system in which students are evaluated for meeting detailed criteria (specifications) for each learning outcome.1 Students either meet the standard for each learning outcome or they do not. Specifications grading can be considered a stepping stone to competency-based education.2 This mastery-based framework prevents students from “masking their deficiencies” by accumulating points in other areas of a course.
The Change from Traditional Grading to Specifications Grading
The course previously used a traditional grading format consisting of 500 points across 3 exams, a capstone project, and a plethora of in-class and asynchronous activities. Students who accumulate 450 points receive an A, 400 points for a B, etc. Theoretically, a student could still earn a B even if they only scored 50% on 2/3 of the exams. I grew tired of that game3 and for various other reasons pertaining to student motivation, grade validity, and the importance of feedback,4 I developed a specifications grading format, as outlined below in Figure 1.
Figure 1 Syllabus Excerpt Describing Course Specifications Grading Schema
Changing the Score
Given that students adopt learning strategies based partially upon grading criteria,2 one question I wanted to answer was “did exam performance change with my specifications grading model?” The only criterion pertaining to exams in my model is that a minimum score of 80% is required on all exams (and/or a remediation exam) to pass the course. The initial impression many have about this criterion is “Won’t students just settle for exam scores in the 80s instead of trying to reach a higher bar?”
Fair question, which is why I analyzed the results.
Did They Settle?
No…and a case can be made that they actually did the opposite. Table 1 displays the results of an unpaired two-sample t-test for each exam from 2022 and 2023 and shows that students under specifications grading had higher average grades on all exams, with significant differences for exams 1 and 2. What makes this more impressive is that the 2022 class had some of the highest grades in course history.
Table 1 Mean Exam Scores by Year
| 2022 (Traditional)N=133 | 2023 (Specifications)N=129 | |
| Exam # | Mean (SD) | Mean (SD) |
| Exam 1 | 93.5 (6.17) | 95.12 (4.59)* |
| Exam 2 | 90.46 (9.56) | 92.79 (9.07)* |
| Exam 3 | 92.18 (7.77) | 93.78 (6.72) |
*Designates significance at P<.05
To obtain a more complete picture, I analyzed the top and bottom deciles to see if there were differences across years for the highest and lowest performing students (Table 2). Students at the top performed as well or better than the previous year and students at the bottom performed better with significantly fewer scoring below 80%.
Table 2 Analysis of Exam Performance by Top and Bottom Deciles
| Top Decile (N=13) | ||||
| Exam # | 2022 Mean (SD) | 2022 Range | 2023 Mean (SD) | 2023 Range |
| Exam 1 | 100.6 (.5) | 100-101 | 101.8 (1.5)* | 100-103 |
| Exam 2 | 101 (0) | 101-101 | 103 (0) | 103-103 |
| Exam 3 | 102.8 (1.2) | 100-104 | 102.5 (1.2) | 100-103 |
| Bottom Decile (N=13) | ||||
| Exam # | 2022 Mean (SD) | 2022 Range | 2023 Mean (SD) | 2023 Range |
| Exam 1 | 80.5 (4.6) | 67-86 | 85.9 (2.1)* | 83-88 |
| Exam 2 | 69.5 (8.5) | 52-78 | 73.4 (8.5) | 63-80 |
| Exam 3 | 75.2 (5.7) | 66-84 | 81 (2.7)* | 75-83 |
*Designates significance at P<.05
What this told me
Basically, the bottom part of the class raised their performance (theoretically because they had to meet the threshold), while the top part maintained high levels of performance. I cannot definitively state that specifications grading caused the outcomes, but the results are what I hoped for. I am also more confident that students learned all the course content at an acceptable level as opposed to simply gathering sufficient points to achieve their desired grade.
Despite a few students having initial concerns with the “all or nothing” 80% standard, they ultimately appreciated the format. Evaluations and anecdotal comments indicated that it freed them from “chasing points,” allowing them to concentrate on learning the material.
Like others, my experience with specifications grading was positive. Even though there are some undescribed nuances of the course and grading model that I will tweak and investigate further, I am definitely staying with specifications grading.
Changes to a teaching or grading system can evoke many different types of questions. What questions came to mind while reading this?
*University of Kentucky IRB deemed this a quality improvement project.
References:
- Nilson LB. Specifications Grading: Restoring Rigor, Motivating Students, and Saving Faculty Time. Stylus Publishing, LLC; 2015
- Townsley M, Schmid D. Alternative grading practices: An entry point for faculty in competency‐based education. J Competency‐Based Educ. 2020;5(3): e01219.
- Cain J. School is a game: Faculty set the rules. Currents Pharm Teach Learn. 2017; 9(3): 341-343.
- Cain J, Medina M, Romanelli F, Persky A. Deficiencies of traditional grading systems and recommendations for the future. Am J Pharm Educ. 2022: Publication ahead of print.
Author Bio(s):
Jeff Cain, EdD, MS is an associate professor and vice-chair in the Department of Pharmacy Practice & Science at the University of Kentucky College of Pharmacy. Jeff’s educational scholarship interests include innovative teaching, digital media, and contemporary issues in higher education. In his free time, he is dad to a pole-vaulting daughter, an extreme trail ultramarathoner, and is president of For Those Who Would, a 501(c)(3) charity in the adventure and endurance racing communities.
Pulses is a scholarly blog supported by a team of pharmacy educators.
Pulses 
Thank you for sharing this. I am actually changing our capstone class here at Manchester to something similar where students have to achieve a passing score on all items to pass the class. I did this following last year when students got 30%s on my calculations exam but still passed the class because there are so many points. Like you say, pharmacists need to be able to do all of these basic skills at a minimum competency. You can’t hit a grand slam in one and expect it to cancel our your single in another.
LikeLiked by 1 person
Quick question: was the 80% threshold to pass in place before moving over to specs grading? I imagine that alone would be a strong incentive for habitual low-performers to “up their game.”
LikeLiked by 1 person
No. Prior to moving to specs grading, it was a traditional points-based system in which they needed to accumulate a total of 450 points for an A, 400 for an A, etc. Yes, that was incentive enough. 🙂
LikeLiked by 1 person