by Ashley Castleberry, PharmD, MAEd
Students do not regularly think about their own thinking; this is especially true regarding exams. Unfortunately, students rarely learn from an exam because they do not often think about why they performed the way they did. Reviewing the exam itself can be a valuable learning experience for students if implemented carefully and intentionally. This post will explore how four schools implemented an exam review process to increase student metacognition. This multi-site project can be a systematic process that could be used in other pharmacy schools or higher education at large.
Did you say “Autopsy”?
To encourage metacognition on exam performance, we created an exam autopsy process. (Yes- autopsy is the correct word here.) Metacognition, simply stated, is thinking about thinking. Metacognitive activities allow learners the opportunity to think about what they know and how it should be applied, as well as plan strategies for learning what they do not know.1 Often, this skill is not intuitive to students.The exam autopsy provides students with a formal process to practice metacognition through reflection on exam performance. Just like medical autopsies, this process helps students to identify what went wrong during their exam. Students review missed questions and categorize the rationale for missed items, identify trends in the types of questions missed, and plan for future learning through reflection on their successes and opportunities for improvement.
Project Objectives
- Develop and implement a multi-campus, collaborative exam autopsy process
- Promote student metacognitive development during the exam review
- Assess the effects of the exam autopsy process on students’ confidence and perceived impact on learning
Who participated?
Participating courses at four institutions used ExamSoft for all in-class assessments administered Spring 2017.
- Northeastern University – Comprehensive Disease Management (P2)
- University of Arkansas for Medical Sciences – Pharmacology II (P2)
- University of Louisiana at Monroe – Integrated Module (P2)
- University of Rhode Island – Integrated Course (P1)
All students (n=481), including any failing any exam throughout the semester, were strongly encouraged to attend voluntary review sessions (mandatory participation for exam score <73% at Northeastern University).
How do you perform an Exam Autopsy?
Students entered a secure environment and were first presented with an overview of metacognitive theory, the exam autopsy process, and the practice of how to use the exam autopsy form. Students reviewed their “Missed Item Reports” and “Strengths and Opportunities Reports” generated by ExamSoft while completing the Exam Autopsy form. The form prompted students to describe missed question content (without copying the item verbatim) and type (multiple choice, fill in the blank, short answer, etc.) while identifying which of the six types of test-taking errors2 they encountered for that item.
Following the review session, which occurred within tens days of the exam, students were invited to complete a voluntary Post-exam Self-assessment via Google Forms which explored student learning / study habits and patterns in the identified errors. Students who did not participate in the group review sessions could schedule alternate times to meet with instructors to review materials. Email addresses were collected on the Google Form to allow instructors to send individualized email reminders to students of their exam autopsy answers. This occurred prior to the next exam to remind them of the study habits/techniques, etc. they said they would change in preparing for the next exam.
What did the students think?
At the end of semester, a perception survey was administered through Qualtrics™. Analysis of course performance with project participation is ongoing but initial perception data was positive. Students agreed or strongly agreed that the exam-autopsy and post assessment survey increased their confidence in preparation for exams (74%) and during the exams (58%).
What did we learn?
Overall, we were pleased with this project and received positive feedback from students at all participating schools. Faculty also liked this process because the scheduled exam review sessions saved time by eliminating multiple one-on-one meetings during office hours, while enhancing collaboration among faculty disciplines during a single review.
During this project we discovered that the exam autopsy report was time-intensive for students – primarily because they were unfamiliar with the process. Although students felt the information from the exam autopsy was useful, they struggled with how to apply the new information to their learning. Unfortunately, student participation in post-exam self-assessment surveys decreased as the semester progressed. This could be due to the time commitment required for survey completion after the review session or lack of perceived value.
What are our next steps?
- Revise our form to include both the autopsy portion and the self-assessment portion
- Create a table for students to reference their error and recommended evidence-based strategies to overcome each error type
- Implement a metacognition inventory to assess impact of the exam autopsy process on student metacognitive development
What metacognitive activities do you use in your curriculum to impact life-long learning?
Acknowledgements
Original research authors include Elizabeth M. Lafitte, PharmD, BCPS (ULM), Michelle L. Caetano, PharmD, BCPS, BCACP, CDOE, CVDOE (URI), Jenny A. Van Amburgh, PharmD, FAPhA, BCACP, CDE (NEU), Adam Pate, PharmD, BCPS (UM).
References
1. Schneider EF, et al. Pharmacy students’ ability to think about thinking. Am J Pharm Ed. 2014;78(8):Article 148.
2. Nolting PD. Winning at Math, Your Guide to Learning Mathematics Through Successful Study Skills. Bradenton, Fla : Academic Success Press, 1997.
Ashley Castleberry, PharmD, MAEd is an Assistant Professor in the Department of Pharmaceutical Sciences at the University of Arkansas for Medical Sciences College of Pharmacy. Educational scholarship interests include metacognition, assessment, and qualitative research. In her free time, Ashley enjoys cooking and teaching her one-year-old son how to talk and walk.
Pulses is a scholarly blog supported by a team of pharmacy education scholars.
Pulses 