key: cord-0835594-icvvk6c8
authors: Eurboonyanun, Chalerm; Wittayapairoch, Jakrapan; Aphinives, Potchavit; Petrusa, Emil; Gee, Denise W.; Phitayakorn, Roy
title: Adaptation to open-book online examination during the COVID-19 pandemic
date: 2020-09-02
journal: J Surg Educ
DOI: 10.1016/j.jsurg.2020.08.046
sha: 1385b394878e96329b7f1e35fb819cd5773a07ac
doc_id: 835594
cord_uid: icvvk6c8

• The COVID-19 pandemic effects medical education worldwide. Program directors have to adapt new assessment format to medical students. Our study shows the effects of online open book examination in multiple choice questions, essay and short answer questions.

COVID-19 altered medical education world-wide. Due to social distancing policies 1 , many medical schools adopted online classrooms and assessments. In Thailand, the outbreak occurred two weeks before the end of the academic year. Course directors at Khon Kaen University School of Medicine discussed an online examination or a proctored, written examination after the outbreak. Our faculty believed this pandemic could persist for several months. Therefore, we decided to create an online examination as postponing the examination could have disruptive effects on subsequent courses.

We created an online final examination for 4th-year medical students in their 12-week surgery clerkship at the time of the outbreak. Students were sent home at the beginning of the 11 th week. Only one week of clinical experiences was affected/cancelled by the pandemic. Since we did not have the technological resources to conduct online monitoring for cheating, we decided that the examination would be open-book. This study aimed to compare students' scores from the online open-book and the traditional closed book examinations in the previous three rotations.

We used our in-house question bank software to deliver an online examination with 120 multiple-choice, six essay, and ten short answer items. This is the same item bank used for pre-pandemic examination, the online test had same number of items of each type as those in the pre-pandemic examination. Students wrote text answers on blank sheets of paper, took pictures of the papers, and then uploaded the pictures electronically when finished. The online examination had the same time limits, examination metrics, and level of difficulty as the closed-book examinations in the previous three rotations. MCQs in all the examinations were taken from the same question bank, however the questions used in each rotation were different. We surveyed all 47 students who took the online open-book examination. Fortyfive medical students (95%) completed the survey.

We compared the percent correct scores across four rotations from 2019-2020 (three traditional in-person, closed book end-of-clerkship surgery examinations and one online, open-book examination). We also looked at correlation between student grade point average (GPA) and examination performance. Continuous variables between four rotations were tested by one-way analysis of variance(ANOVA). A linear association between two variables was tested by Pearson's correlation coefficient. Fisher's z-test was used to compare two correlation coefficients.

Compared with the three traditional groups, medical students who took the online, open-book examination had a higher mean score in both MCQ and essay examinations, but a lower mean score in the short-answer examination ( Table 1) . These results were different to previous published studies which demonstrated equivalence between open and close book examination. 2, 3 Interestingly, the online, open-book examination group had a significantly lower correlation between the essay scores ( Table 2 ) and their GPA than the three previous traditional groups (r=0.191). However, the correlation between the MCQ item scores and short answer item scores with prior GPAs in each clerkship was not statistically significant. These results may indicate that students with lower GPAs may benefit more from the openbook format such they were able to do better on these items than prior GPA predicts. We did not collect data about what types of reference materials were used by the students to assist in answering the test questions.

Surprisingly, medical students preferred a traditional closed-book examination over an online, open-book examination (62.2%). However, students generally believed (66.7%) that our approach was the best solution to prevent contagion during the COVID-19 pandemic.

Although an online, open-book examination was feasible during the COVID-19 pandemic, this study indicates that mean MCQ and essay scores may be higher and short answer scores lower than for closed book examinations. Short answer scores had conversely higher correlation with GPAs than the others. This result is important for setting a comparable pass mark for closed and open-book examinations. And should be repeated at other institutions who have switched examination formats. Also, clerkships may want to use data from several open-book examinations and recalculate passing scores prior to assigning final grades. Further research is required to assess the downstream effects of online, open-book examinations on long-term knowledge retention and application. 

Medical Student Education in the Time of COVID-19

Comparing Open-Book and Closed-Book Examinations: A Systematic Review

Influence of open-and closed-book tests on medical students' learning approaches