key: cord-0786660-8hhj7wc7
authors: Bracken, Keyna; Levinson, Anthony J.; Mahmud, Meera; Allice, Ilana; Vanstone, Meredith; Grierson, Lawrence
title: Spiralling Pre-clerkship Concepts into the Clinical Phase: Augmenting Knowledge Transfer Using Innovative Technology-Enhanced Curriculum Activities
date: 2021-06-22
journal: Med Sci Educ
DOI: 10.1007/s40670-021-01348-1
sha: 57f3869d2728b2e0496aa8c1afcc646de3822217
doc_id: 786660
cord_uid: 8hhj7wc7

BACKGROUND: Clerkship is a challenging transition for medical students where they learn to apply functional knowledge and diagnostic reasoning skills learned in the pre–clinical phase into the clinical environment. Rather than a smooth continuum to facilitate application of knowledge, clerkship blocks are discrete, fragmented structures with little integration. Developments in cognitive psychology and increasing attention to the student learning environment are driving more purposeful integration in medical education. We sought to enhance knowledge transfer in the Family Medicine clerkship by developing an e-learning pathway with both asynchronous and synchronous components to integrate pre-clerkship problem-based learning (PBL) cases into more complex clinical scenarios. METHODS: A parallel-convergent mixed methods evaluation was conducted, which included comparison of learning outcomes (exit exams) relative to the prior class, knowledge check quiz performance, and qualitative analysis of student and faculty perceptions. RESULTS: Analyses revealed no significant difference between exit exam scores of the intervention and pre-intervention clerks (p = 0.30). There were statistically significant differences in mean quiz scores over the rotation (p = 0.0001). Moreover, learners and faculty each perceived the integration components as facilitating the transfer of pre-clinical learning into clerkship activities. CONCLUSION: The novel e-learning pathway firmly anchored FM clerkship learning and will continue to ensure learners are ideally primed to optimize their direct clinical learning opportunities.

and physical classrooms, into real-life clinical care situations [1] . Clerkship is most frequently organized as a series of clinical rotations that highlight each of the core medical disciplines. The transition from preclinical to clinical training can be challenging for medical students for several reasons: they must transition to a number of different clinical learning environments [2] , navigate new hierarchical learning structures [3] , contribute to patient care, all while simultaneously understanding and meeting the expectations of frequently changing preceptors [4] [5] [6] [7] . To support student learning through this transition, clerkship activities in most core medical disciplines involve discipline-specific tutorials designed to stimulate students' acquired knowledge in a way that helps them apply the core principles learned in the preclinical phase in the context of the relevant clinical environment [8] . Notably, these tutorials are typically organized and developed within the bounds of discipline-specific knowledge without consideration for the totality of students` clerkship experiences. Accordingly, they are regularly experienced by students as discrete and fragmented learning opportunities, which are distinct from their clinical activities and so do not facilitate the development of coherent and integrated medical knowledge [3] .

In 2005, the Undergraduate MD (UGME) Program at the Michael G. DeGroote School of Medicine at McMaster University (Hamilton, Canada) aimed to better integrate learning across the pre-clinical and clinical phases (clerkship) with its innovative COMPASS curriculum [9] . The acronym dissected with liberal poetic license, means-Concept Oriented, Multidisciplinary, Problem based, Practice for transfer, Simulation and Streaming [9] . The COMPASS curriculum, building on McMaster's trademark Problem Based Learning (PBL), was structured to provide horizontal and vertical integration of fundamental basic science into problem-based clinical examples [3] . Horizontal integration considers the way in which several to-be-learned topic areas are presented with respect to one another. Horizontal integration means that material related to a single topic or subject is delivered in parallel with other topics, taught in multiple ways concurrently and with the same level of complexity. Vertical integration, on the other hand, leverages ideas from adult learning theory and cognitive psychology to promote the integration of knowledge [10, 11] . Specifically, the vertical approach builds on the initial foundation of knowledge in any one topic area and presents the materials with increased complexity so that learners gradually develop more capacity for applying acquired knowledge into novel and more difficult contexts [1, 10, 12] . When combined effectively, horizontal and vertical integration are understood together as "spiral integration," with defined topics and subject areas interwoven to help learners develop a more comprehensive understanding of the collective knowledge [10, 11] .

While many components of COMPASS were integrated horizontally across the pre-clerkship phase with success, others proved more challenging; notably, the practice-totransfer and simulation components. The promise of vertical integration was also not fully realized, as there were few opportunities to apply increasingly complex knowledge in clerkship, and there was no opportunity to revisit foundational concepts with diverse clinical scenarios [13] . The addition of two distributed campuses, Kitchener-Waterloo (KW) in 2007 and Niagara in 2008, added additional difficulty around ensuring curricular equivalency at all teaching sites. Clerkship tutorials, similar to pre-clinical phase tutorials, are meant to facilitate the application of foundational knowledge to novel clinical situations. However, the lack of integrated curriculum, variability of teaching effectiveness amongst tutors, and the challenge of delivering equivalent face-to-face tutorial experiences across three distributed campuses made for a discrepant and potentially suboptimal experience for students. Given some of these issues, such as the newly distributed landscape, in addition to updates in medical and healthcare education systems, a multi-year process of curriculum renewal was undertaken with gradual rollout late in 2019. Here, COMPASS was modified with the aim to "more efficiently build, consolidate and integrate … knowledge and skills, thus offering an opportunity to measurably enhance students' clinical transfer of concepts introduced in the pre-clinical curriculum" [9] .

Accordingly, we dissected the various curricular components of COMPASS, with the aim of developing a series of new rotation-specific curricula. We began this exercise with a focus on improving the Family Medicine clerkship.

Specifically, we identified and emphasized core family medicine concepts reflected in multi-disciplinary problembased learning cases from pre-clerkship, spiraling them from simple to more complex and revisiting them in clerkship. To enhance knowledge transfer, we referenced evidence that supports popular cognitive psychology learning theory and introduced multiple opportunities for retrieval-based practice, which employed well-established blocked, spaced, and interleaved presentation techniques [14, 15] . Further, informed by adult learning theory, we added interactive multimedia e-learning experiences to broaden exposure to clinical problems that may not be encountered in a clinical rotation and to optimize knowledge retention [16, 17] . This latter innovation involved a blended teaching approach that included synchronous and asynchronous online instruction. Students engaged with asynchronous e-learning modules housed in the Undergraduate MD Program's learning management system (LMS), which were followed by synchronous faculty-led sessions delivered to all campuses via a web-based video-conferencing platform.

This curricular intervention took the form of the "Family Medicine Clerkship e-Learning Pathway" (see Appendix 1) with the following modular components:

Asynchronous Content Five PBL cases, which were previously encountered during pre-clerkship, were purposefully selected for inclusion in the new curriculum These cases represent key health themes throughout the lifespan-Pediatrics, Care of the Adolescent, Care of the Adult, Care of Elderly, and Palliative Care-and served as individual modules for this asynchronous content. The education pathway required the clerks to work through these adapted PBL cases, which were modified to present additional complexity relative to their original presentation. This encouraged application of foundational concepts into novel clinical situations. Each module was designed to emphasize core content, in preparation for the associated synchronous tutorial.

At designated times each week during the clerkship rotation, students participated in a 3-h synchronous tutorial that was facilitated by faculty leaders -also known as "maestros"-who were both subject and content experts in that week's topics. Tutorials were conducted using a web conferencing platform (WebEx, Cisco Systems, California, USA) with the maestro remote from all the student groups. As this was pre-pandemic, students would meet in-person, in small groups (maximum eight students), to participate in the web conferences: for example, one small group in Niagara and KW, and two in Hamilton. The maestro joined from their own remote location. During tutorial, the asynchronous module content was reviewed, provocative cases were discussed, and important key concepts were articulated. Both the asynchronous modules and synchronous tutorials addressed the inherent challenge of providing an equivalent learning experience across all campuses.

Assessment Embedded within the modules were knowledge check exercises. These took the form of multiple-choice quizzes comprised of 8 questions, which were selected randomly from an existing formative assessment bank and mapped to core family medicine clerkship topics. This bank was developed by the Canadian Family Medicine Education Directors (CUFMED) division of the College of Family Physicians of Canada (CFPC). These MCQs were administered pre-, mid-, and post-rotation and functioned as a type of formative progressive performance index. Further, each PBL module concluded with a Concept Application Exercise (CAE), which required a narrative essay-type short answer response to questions designed to encourage the integration and application of knowledge [13, 18, 19] . Individual feedback was provided to the students by the maestros.

Lastly, an established exit exam comprised of fifteen MCQ questions based on clinical decision making was administered electronically using the ExamSoft (ExamSoft Worldwide LLC, Texas, USA) platform within the final days of the clerkship rotation. This assessment is consistent with that administered prior to this curriculum redesign.

A complete overview of the innovative clerkship curriculum, linked with learning objectives, is shown in Appendix 1.

In this report, we present an evaluation of this new Family Medicine Clerkship e-learning pathway. Our goals were to evaluate the impact of the e-learning pathway on student learning and to understand student and faculty perceptions of the learning pathway and its implementation. We anticipated that the new curriculum intervention would increase students' ability to apply foundational knowledge in different clinical situations (i.e., knowledge transfer), and that such learning would facilitate better exit exam performance. We also reasoned that the integrated online blended teaching approach would be perceived favorably by students and faculty and refine clerkship curriculum for the future.

This study was carried out using a three-armed, parallel, convergent, mixed-methods study design [20] , which included quantitative review of student performance on tests of knowledge and qualitative analysis of interviews with students and faculty to ascertain their perceptions of the new e-learning pathway. This integrated approach was chosen for its ability to offer multiple forms of evidence as to the impact of the curriculum on student learning and student experience, as well as an opportunity to appraise the features of experiences that impacts the learning. Results are reported informed by the Good Reporting of a Mixed Methods Study (GRAMMS) framework. These analyses were conducted separately and integrated at the end of the study [21] .

The first and second arms of the study quantitatively compared (1) in-progress and (2) final exam scores using results from (1) a multiple-choice question quiz administered before, during and after clerkship, and (2) final exam scores between students who participated in this innovative clerkship curriculum versus students from a previous cohort, before the curriculum was introduced. The third arm employed a qualitative descriptive approach [22, 23] and individual interviews to understand the experiences of students and maestros.

This study was conducted at the Hamilton, Kitchener-Waterloo and Niagara UGME campuses of McMaster University (Hamilton, Canada). Research Ethics Board approval was obtained from the Hamilton Integrated Research Ethics Board. All medical students entering their Family Medicine Clerkship rotation at all three UGME campuses between May 2019 and May 2020 were informed about the study via a recruitment email, an informational flyer which was sent by email, printed in their Family Medicine Clerkship Information Package, and added to their electronic welcome package on the local LMS, and through an in-person presentation. During these presentations, informed consent forms were distributed to students for future participation in an interview after clerkship.

Of the 208 undergraduate medical students in the medical school clerkship, 160 students participated in the new innovative Family Medicine clerkship curriculum, dependent on the timing of their Family Medicine clerkship core. Scores from all participating clerks were analyzed for the first arm of the study. The UGME Program at McMaster University provided the exit exam scores of the previous class; exit exam scores of 119 undergraduate medical clerks from the 2019 class were analyzed for the second arm of the study. For the third, qualitative arm, the concept of "information power" was utilized to guide the determination of sample size, where interviews were conducted with consenting students who completed the new curriculum and until enough information was present to adequately achieve our objective [24] . Nine students and five maestros participated in this phase.

In order to achieve the first objective, a multiple-choice quiz (MCQ) comprised of 8 questions, derived from the previously described question bank, was administered through the LMS at three specific points throughout the clerkship rotation. The quizzes were randomized to ensure questions from pediatric, adult, and geriatric domains were represented in the same proportion at each offering. The MCQs were taken pre-, mid-, and post-rotation. The questions supporting these tests were derived from an existing formative assessment bank developed by the Canadian Undergraduate Family Medicine Education Directors (CUFMED) division of the College of Family Physicians of Canada (CFPC). Student scores were stored in the local LMS and were analyzed by the research team.

The MCQ quizzes were limited to a single attempt. There was a technical issue for the first group, which allowed a few students (n = 4) to complete more than one attempt on the pretest and mid-rotation quiz; however, the data from their first attempt was used in the analysis. This technical issue was fixed for all subsequent quizzes and did not affect any of the other groups.

The exit exam scores, the other component of student performance, were compared with the pre-intervention class.

To understand students' experiences and perceptions of the learning pathway, as well as the impact of this curriculum on their clinical performance, semi-structured interviews were conducted across all clerkship sites among students with a variety of achievement levels on the in-progress quizzes. Maestros were also purposively sampled to include a variety of experience levels in order to explore the impact of the new curriculum on the educators' role and on the perceived quality of educational experience provided. Interviews took place in person or via video conferencing platform and were transcribed verbatim. Interviews were audio recorded and lasted about 30-45 min each, with honesty and confidentiality emphasized. Quantitative and qualitative component results were integrated and compared to generate conclusions [21] .

All statistical analyses for the quantitative data were performed using Stata Software Package, version 14.2 (Stata-Corp LLC, Texas, USA). Significance was defined as having a p-value of < 0.05, all p-values were double sided, and 95% confidence intervals were computed.

For the first study arm, the results of the MCQ pre-, mid-, and post-rotation quizzes were compared in a one-way repeated measure analysis of variance (ANOVA) to understand whether there was a difference in test scores amongst students before, during, and after the innovative clerkship curriculum. In addition, the mean scores, and standard errors were computed for each group's MCQ score. Tukey's HSD post hoc methodology was utilized to indicate where the significant effects occurred between the three test groups.

To compare student performance in the innovation group to the previous class, exit exam scores were evaluated using a one-way ANOVA with Group (Pre-intervention clerks, Post-intervention clerks) as the only factor. These analyses determined whether there was a significant difference in mean exit exam scores between the two groups. Summary statistics of the exit exam scores for both groups were also estimated, providing mean scores and standard errors.

In order to determine and understand the perceptions of the new clerkship curriculum, semi-structured interviews were conducted with both students and maestros. Interviews were analyzed using an inductive content analysis approach [25] . The intent of this analysis was to develop a rich description of learner and maestro views, which offers both a summative and an integrative element across participants. Initial coding began by analysts coding independently, then meeting to discuss and create a refined coding schema, (Appendix 2) which was then applied to all interviews. N-Vivo 12 was used to manage qualitative data analysis.

MCQ tests (pre-rotation, mid-rotation, and post-rotation) were completed and assembled for six groups; group seven did not participate due to the 2020 Covid-19 pandemic's disruption of their clerkship rotation. A total of 142 of the 160 students involved in the clerkship completed the MCQs; therefore, MCQ scores were reviewed and analyzed for these individuals.

Results showed that the innovative clerkship curriculum elicited statistically significant differences in mean MCQ test scores over its time course (F (2, 423) = 9.10, p = 0.0001; Table 1 ). Furthermore, post hoc comparisons using the Tukey HSD test indicated that clerks' MCQ performance at Pre-rotation (M = 61.54%, SE = 1.67) was significantly lower than at Mid-rotation (M = 69.39%, SE = 1.49) and Post-rotation (M = 69.20%, SE = 1.26). However, mean MCQ scores did not significantly differ between Mid-rotation and Postrotation (Table 2 ).

A total of 119 students' exit exam scores were analyzed from the previous class of Family Medicine clerks (class of 2019) against the students' exit exam scores from the new clerkship curriculum (N = 160). The mean exit exam score was higher among the intervention group (M = 74.77%, SE = 0.56) than the controls (M = 73.96%, SE = 0.49); however, there was no significant difference between the exit exam scores of the intervention group compared to the previous group (F (1, 277) = 1.10, p = 0.2958, Table 3 ).

Overall, there were five maestros interviewed from the new Family Medicine clerkship, and a total of nine students from five groups, all of whom completed their clerkship between September-December 2019. Participants discussed their perceptions of content, format, tutorials and assessment.

Students shared both positive and negative feedback about the content of the e-learning pathway. In many examples, the contribution of the tutorial discussion to positive appraisal of content was related to the presence of complex issues of care such as social determinants of health, de-prescribing, or appraising guidelines for application to a specific patient. Participants frequently remarked that they appreciated the presence of these issues in the e-modules and benefitted strongly from discussion about applying these principles in practice.

Participants were resounding in their emphasis on the need for content to be relevant to clinical care and outlined the importance of providing concrete strategies and practical skills. They provided many examples of using the content from the COMPASS curriculum in later clinical experiences, and although students appreciated the practical content that did exist, they wanted more:

I was also expecting to learn more practical things that you would really need to know as a family doctor, but I don't think we got so much of that. … we didn't really talk about if you see a patient like this, this is how you prescribe, this is how you fill out this form or refer someone to this other specialist. … (Student #5)

Maestros expressed their satisfaction with the material that was covered in the tutorials and appreciated students' feedback on topics discussed. Several maestros had a positive perception on the complexity of pre-clerkship cases which were reviewed, mentioning that they stimulated students to think differently and enhanced their learning process.

Overall, students had more positive than negative opinions about the structure of e-modules, with emphasis placed on the helpful ways e-modules prepared them for tutorials. Participants also appreciated the practice questions and concept application exercises provided in the learning pathway, which were recognized as being useful in addressing gaps of knowledge by most students:

And, also, the concept, so the CAE (Concept Application Exercises) that you mentioned, like the exercises that we got to do, I think that was really, really helpful. I really liked that part of that, like that really forced me to look into the topics, to think like systematically and, yeah, I think that was really helpful. (Student #6)

Students provided considerable feedback about the format of tutorials which initially were plagued with technical issues relating to a remote technology delivery system that few students or faculty had experience with. At the time they encountered the learning pathway, most material was delivered in-person and, pre-pandemic, there was vastly less experience and expertise with web conferencing technologies.

In many examples, the negative perception of the tutorial format was related to gaps in communication between maestros and students. Participants frequently remarked that there were many awkward silences, leading to distractions and wasted time: Maestros also expressed the challenges they faced when communicating with students: "I still struggle. I don't know what's the right amount of talking and the right amount of texting. That is still a struggle and I haven't figured out that balance." (Maestro #3) Some maestros expressed their difficulty with using the features of the online platform to optimize communication; however, in most cases, maestros were able to gauge what worked best.

Participants appreciated when maestros facilitated discussions and structured their tutorials as being discussion-based as opposed to a didactic lecture format. For example, one participant stated:

One of the times that I found tutorials very helpful, when they were actually encouraging us to have a discussion. When they weren't and it was very didactic, that was not helpful at all, because we all sort of tuned out at that point. (Student #5) When discussing their opinions on the synchronous format of tutorials, a majority of students reported that they perceived them as being beneficial for their learning. Several participants stated that this format helped "dedicate time to spend learning about the topics" (Student #5) and enhanced their learning because of "pretty engaging discussions all around." (Student #8) Many participants mentioned that their classmates' contribution was important in increasing their knowledge about different aspects of Family Medicine.

Participants expressed frustration of having to connect remotely to a maestro, with the technical challenges often presented. However, participants also addressed the concept of ensuring equity of teaching for students from regional campuses, as they noted that pre-pandemic, almost all teaching sessions, either pre-clerkship or clerkship, were "broadcast" from Hamilton, creating an unequal experience for students at the other campuses. Maestros also commented on the advantages of the new tutorial system in providing uniform learning to all students:

From an equity perspective for the students who previously may have felt that they're not quite getting the same kind of education experience as the ones who are physically in the room with the tutors, or you know, making sure that the different sites have the same education experience, because you know, even an in-person tutorial can be varied depending on who's teaching it and how it's organized. So, I think from that perspective, McMaster especially with having regional campuses, I think that's a potential advantage there for the students. (Maestro #4) Interestingly, both students and maestros commented on the potential for student-led tutorials remotely facilitated by a faculty preceptor as a way of promoting student engagement.

Maestros' delivery style appeared to impact students' perception of tutorial. Aspects of maestros' teaching styles that were praised by participants included prompting in-class debates, facilitating discussions, encouraging student-led presentations for the discussion of important topics or for answering questions, and providing feedback and key summaries at the end of tutorials. One participant provided a distinct example of a stimulating tutorial, in which they lauded the maestro's teaching style:

So, one of the maestros did a really, really good job of having cases. And I'm trying to remember which tutorial it was, but they had really good case examples, that it was sort of you go through the case, they make us answer it in our groups, and then we review it, so that they call on us to review and they give us sort of like the main points to take away. I really liked that structure because it not only made us all work together to try to think of an answer, but then it also made sure everyone was engaged because it held us accountable and it asked us, so what was your answer. (Student #9) Due to the online format of tutorials, participants acknowledged the challenges associated with approaching maestros to discuss personal experiences: "I didn't feel that I could approach the tutorial leader at all because I was in a room full of other… because they didn't really know me, I didn't really know them, they were just kind of this face on the screen, so it's a little depersonalized." (Student #5) Participants expressed their appreciation of maestros who offered support and structured tutorials so students would feel comfortable talking about their personal challenges. Students reported that this support and guidance also helped them grow as a medical student, to "develop reasoning and judgement" (Student #8) and made them comfortable asking questions.

Participants appreciated the formative multiple-choice questions (MCQ), stating that they were useful, "it makes me want to review the information that was tested." (Student #9). Participants commonly commented on ways in which the MCQ enhanced their learning: Positive appraisal about the assessment was frequently linked to the Concept Appraisal Exercises (CAEs). In many cases, participants mentioned how effective these exercises were in helping them think analytically and encouraged them to review topics on their own time:

But the post ones, the ones that were much longer were definitely helpful.…I just appreciated having those questions, was like a prompt for learning about that topic and because they were hard enough and I felt challenged, I found them interesting. (Student #9)

This project aimed to create and evaluate an innovative Family Medicine clerkship curriculum, designed as an e-learning pathway with asynchronous e-modules using existing pre-clerkship case-based problems, interwoven with additional layers of complexity to encourage application of core foundational concepts into different clinical scenarios. The modules spanned the lifecycle; incorporated a series of knowledge check assessments that assisted with student identification of learning gaps and progress; engaged students with multimedia, interactivity, and concept appraisal exercises; and were bolstered by a series of scheduled synchronous tutorials facilitated by expert faculty.

The results show that the curriculum had a positive influence on student learning. Review of the MCQ quiz scores shows that students' knowledge improved through engagement with the e-learning pathway and clinical exposures that occurred as part of the clerkship rotation. These data show that students had the largest gain in knowledge between pre-intervention and roughly the mid-way point of the rotation, with little apparent improvement from midto-end of the rotation. This might be explained by a few different factors. First, we must acknowledge that the quiz items may have been overly challenging, creating a plateau on performance improvement. Second, the mid-rotation quiz occurred slightly later than mid-rotation; with students having completed 3/5 of the rotation. Thus, they would have had more instructional interventions (curriculum) and clinical experiences between pre-rotation to mid-rotation than from mid-to end-rotation. With respect to learning relative to previous classes, we noted no significant difference between exit exam scores between students who underwent the innovative e-learning system compared with the previous class. This finding highlights, at least, a non-inferiority outcome for the new curriculum.

Our qualitative analyses of student and faculty perceptions highlighted the influence of greater curriculum integration on student readiness to learn, clinical capability, and provided the foundation for more robust assessment of student competency. Overall, both students and maestros perceived the e-learning pathway as being beneficial to student knowledge translation. Students perceived the CAE as the most helpful form of formative assessment; perhaps due to the written feedback provided, although the knowledge check quiz questions were also perceived as useful. The e-modules were generally found to provide helpful material for tutorials and useful to revisit. Negative perceptions of the synchronous web conferenced tutorials most commonly related to technical difficulties, and lack of opportunity for meaningful discussion due to dead space and the remote tutorial format. Despite this, students did appreciate the opportunity for meaningful discussion with their classmates and appreciated the equitable access to the maestro across all sites. Notably, at this time, we realize the technical challenges experienced are mostly moot; since the pandemic, videoconferencing platforms have become the norm rather than the exception, and both student and faculty comfort with remote technologies and virtual discussions has increased.

In integrating the quantitative and qualitative results, we are confident that learning occurred over the course of the Family Medicine clerkship; this is evidenced in faculty reports of student performance, the improved MCQ outcomes, and the non-inferiority of the exit exam scores compared to the pre-intervention cohort. The fact that we were not able to demonstrate statistically significant improvement in terms of our first objective may be influenced by several factors. With respect to the MCQ quizzes, the small number of test items (eight) was a pragmatic choice to avoid testing fatigue. Although the questions were derived from a reliable question bank reviewed and improved by two expert MCQ writers, the bank has not been the subject of rigorous psychometric review. As the questions for each quiz were randomly selected from question pools, it was impossible to ensure the same level of difficulty. The exit exam-while not the subject of reliability and variability studies-has been evaluated and subjected to robust in-house psychometrics. We should reasonably accept effective and non-inferior student learning as an outcome for this study, as it is relevant that students within the intervention group performed as well as their peers who were not. The interviews were all recorded near the end of project completion and subject to recall distortion and depending on timing of the clerkship rotation, may have been negatively skewed by the earliest group's technological difficulties. The last group was not able to participate due to the pandemic, as medical undergraduate learners were abruptly removed from clinical learning environments March 15, 2020.

The novel Family Medicine clerkship afforded students more opportunities for knowledge translation and better met the intended goals of the revised COMPASS curriculum. The positive effects on learning and participant perceptions justify the success of our curricular intervention, which is equivalent across all campuses and easily scalable to other clerkship disciplines. Opportunities to provide high quality, educationally robust, asynchronous, technology-enhanced learning have become critical as medical schools rapidly pivoted away from in-person group instruction and limit direct clinical experience on account of the Covid-19 pandemic. Indeed, we realized additional benefits of this e-learning pathway when clinical learning environments were curtailed in 2020. There is room for further study to understand the impact of this pivot.

We envision the development of similar spirally integrated, e-learning pathways across clerkship disciplines. In doing so, we aim to offset some of the largest challenges with transitions to clerkship. We anticipate that this will have demonstrably positive impact on long-term knowledge retention, clinical experiences, and performance on instrumental licensing exams. Preparation for future FM work Comments relating to how they felt COMPASS prepared them to proceed to future FM work (which may include clerkship or electives or residency training. It may also refer to FM-related work they may do in another specialty) Regional campus perspectives

The issue of equity and participation across different campuses seems particularly important Student traits and previous experiences Interest in FM (pre-clerkship and after clerkship) Placement of FM core within clerkship Previous clerkship experiences outside of FM Resources used for self-improvement Students often talk about how FM fell within their clerkship year (first or last rotation so was or was not so sufficient) changed their experience of the curriculum (previous experiences outside of FM). This will also capture talk about how interest and experience with FM changed their experience of the curriculum Suggestions for improvement For concrete suggestions or requests for changes to the program Synchronicity

The question of synchronous vs. asynchronous aspects seems important and not always tied to negative or positive feedback Online v in person

The issue of being together in person or online (and varying combinations of this (e.g., students in one room, tutor online) can be captured here. Much of this talk will also probably get categorized as positive or negative feedback/experiences around the tutorial, but I think it's useful to keep it in one spot Topics covered Negative perspectives Positive perspectives Since topics covered in tutorials and e-modules were relatively similar, pulled content/topic out as its own code. This includes comments related to topics that they would have liked to discuss more, and topics which they greatly enjoyed and perceived as the most useful for learning

Name of code Description Perception on content Positive Negative Description of learners and maestros' opinions on the concepts covered and discussed in e-modules and tutorials, also includes perceptions/ comments on the transition of knowledge/content to clinical practice Perception on format Positive Negative Description of how learners and maestros perceived the content was delivered for e-modules and tutorials (including online vs. in person, synchronicity and regional campus perspectives) Perception of maestros/tutorial leaders Positive Negative Comments from learners relating to the positive and negative appraisal of maestros in terms of their teaching styles, level of engagement and involvement, effort in connecting with learners etc Perception of assessment evaluations Positive Negative Learners perception of assessment evaluations presented throughout e-modules and tutorials (e-modules MCQ, CAEs)

Relationship between COMPASS and interest in FM Speaking to how well the COMPASS curriculum influenced learners' perception and interest in the FM clerkship and discipline

Ethics Approval Research Ethics Board approval was obtained from the Hamilton Integrated Research Ethics Board (HIREB #5482). The procedures used in this study adhere to the tenets of the Declaration of Helsinki.

Informed consent was obtained from all individual participants included in the study.

A strategy for teaching and learning in the PBL clerkship

Opportunity or threat: the ambiguity of the consequences of transitions in medical education

The integrated curriculum in medical education: AMEE Guide No 96

Medical curriculum training and the continuum of medical education

Exploring the transition of undergraduate medical students into a clinical clerkship using organizational socialization theory

Medical student strategies for actively negotiating hierarchy in the clinical environment

The relationship between perceived preceptor power use and student empowerment during clerkship rotations: a study of hidden curriculum

PBL in the Undergraduate MD Program at McMaster University: Three Iterations in Three Decades

Our Curriculum

How learning works: Seven research-based principles for smart teaching

What is a spiral curriculum?

Planning and implementing an undergraduate medical curriculum: The lessons learned

Deconstructing integration: a framework for the rational application of integration as a guiding curricular strategy

Improving students' learning with effective learning techniques: Promising directions from cognitive and educational psychology

Make it stick: the science of successful learning

Cambridge: Cambridge University Press

Teaching genetics with multimedia results in better acquisition of knowledge and improvement in comprehension

Methods to assess students' acquisition application and integration of basic science knowledge in an innovative competency-based curriculum

Evaluation of essay questions used to assess medical students' application and integration of basic and clinical science knowledge

Research design: qualitative quantitative and mixed methods approaches

The quality of mixed methods studies in health services research

Whatever happened to qualitative description?

What's in a name? Qualitative description revisited Res Nurs Health

Sample size in qualitative interview studies: guided by information power

Three approaches to qualitative content analysis

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

We would like to acknowledge the early contribution of many collaborators early in the case development phase of this work. In particular, medical students at the time, now residents in 

 Name