Exams at Home: COVID-19 Required, Student Approved

By: Shelby Humpert, PharmD Candidate 2021; Ashley Castleberry, PharmD, MEd; Renee’ Acosta, RPh, MS

The 2020 Spring semester will not be forgotten anytime soon. Most universities cancelled in-person classes in March, beginning a time of chaos for administrators, faculty, and students. The COVID-19 pandemic brought expeditious change to The University of Texas at Austin College of Pharmacy, including a rapid shift to online learning and remote examinations using ExamSoft’s ExamMonitor feature. Previously, student pharmacists took exams on their personal computers onsite with ExamSoft, but the pandemic forced the move from in-person to at-home testing. There were no changes made to exam content, number of questions, or amount of time for each exam. There was essentially no difference in resource allocation for faculty and staff because the time required to review flagged incidents was comparable to the time spent proctoring in-person exams. Financial resources increased substantially to account for the proctoring service provided by ExamMonitor.  

Has remote testing been used before?

There is little research on remote examinations in pharmacy education. Many previous studies about this topic evaluate online versus paper testing, the effects of online proctoring, or academic integrity concerns.¹ One such paper discussed that 80% of students felt more uncomfortable being “watched” with remote proctoring than in person.²  However, another stated that even though students did not like the proctoring side of exams, the overall remote testing experience was positive.³ 

At the end of the Spring semester, we conducted a survey to evaluate the online, test-from-home experience of first- and second-year student pharmacists to determine its impact and help inform the college’s examination policies in the future. 

Areas of interest included:

• Ease of transition to remote testing
• Remote testing environments
• Concentration
• Stress and anxiety
• Assessment of knowledge 
• Academic dishonesty 
• Testing preferences for future semesters 

Everyone likes surveys, right?

A 30-item survey was developed and tested for face validity by selected students. The survey was administered via Qualtrics following 7 weeks of remote learning and testing. The survey asked about a variety of topics related to the testing process, however we decided to focus those questions pertaining to student experience and preferences (see Figure 1). 

Here is how students felt:

A total of 128 out of 237 responses were received; 114 students completed the survey (48% response rate). More responses were received from first-year students (59%) compared to second-year students (41%). There were no differences between the classes for testing experience. 

Figure 1: Remote Examination Experiences

Questions: 

1. It was easy for me to transition to remote testing.
2. It was easier for me to concentrate on the exam with remote testing compared to in-person testing.
3. I was able to take my exam in a low distraction environment.
4. I felt less stressed taking exams remotely than in-person.
5. I felt less anxious taking exams remotely than in-person.
6. Overall, I feel my knowledge is being evaluated accurately with remote testing.
7. Academic dishonesty is more likely to occur with remote testing than in-person testing.

For examination preferences, 79% of students reported a preference to test from home and 41% reported a preference for exams in the afternoon over the morning (35%) or evening (24%). Eighty-six percent selected preferences for remote testing even if in-person classes resume; however, 47% stated they want the option for in-person testing accommodations. While the majority preferred afternoon exams, first-year students reported a secondary preference for evening exams while second-year students reported a secondary preference for morning exams.

A big concern with remote testing is exam integrity. Only 3.5% of students reported knowledge of any honor code violations that took place with remote testing. When comparing exam grade averages from Spring 2019 to Spring 2020, no significant differences were seen. Only two years of data is available since the implementation of ExamSoft at our college and is from different cohorts of students, limiting the quality of the comparison. 

The results of this study support the use of remote examinations for PharmD students. Based on survey responses, students had an overall positive experience with remote testing. Our college of pharmacy decided to continue with remote exams for the Fall 2020 semester; however, we are allowing students to opt-in to testing on campus with enforced COVID-19 safety measures. We had 10 first-year, and 1 second- and third- year students opt-it to test on campus Fall 2020. We also scheduled final exams during the afternoon based on the students’ testing time preferences. 

Pandemics are stressful and so are exams. Shouldn’t colleges of pharmacy do their best to create an environment that promotes well-being regardless of whether or not a pandemic dictates the scenario? Other colleges of pharmacy should consider these results when determining examination policies for their student pharmacists in an effort to improve testing experiences in both virtual and non-virtual learning environments.

What challenges have you had with remote testing?

References:

1. Cramp J, Medlin JF, Lake P, et al. Lessons learned from implementing remotely invigilated online exams. J Uni Teach Learn Pract 2019;16(1),10.
2. Mirza N, Staples E. Webcam as a new invigilation method: students’ comfort and potential for cheating. J Nurs Educ 2010;49(2), 116-119.
3. Weiner JA, Hurtz GM. A comparative study of online remote proctored versus onsite proctored high-stakes exams. Journal of Applied Testing Technology 2017;18(1),13-20.

---

Author Bio():

Renee’ Acosta, RPh, MS is a Clinical Professor in the Division of Pharmacy Practice and the Associate Dean for Academic Affairs at The University of Texas at Austin College of Pharmacy. Educational scholarship interests include analysis of admissions, academic performance and academic progression analysis. In her free time, Renee’ enjoys spending time with her family and being outdoors.

---

Pulses is a scholarly blog supported by a team of pharmacy education scholars