key: cord-0881102-rolztyxg authors: Hadvani, Teena; Dutta, Ankhi; Choy, Eric; Kumar, Shelley; Molleda, Carolina; Parikh, Vipul; Lopez, Michelle A.; Lui, Karen; Ban, Kathryn; Wallace, Sowdhamini S. title: Effectiveness of Modalities to Teach Evidence Based Medicine to Pediatric Clerkship Students: a Randomized Controlled Trial date: 2020-09-30 journal: Acad Pediatr DOI: 10.1016/j.acap.2020.09.012 sha: 83ad30c18fbc803ef22f87b6732c220ead1aed30 doc_id: 881102 cord_uid: rolztyxg OBJECTIVE: To evaluate the effectiveness of a traditional didactic session (TDS) as compared to a self-paced, interactive, multimedia module (SPM) on the application of evidence-based medicine (EBM) skills among medical students during their inpatient pediatric rotation. METHODS: We conducted a randomized controlled trial from 6/2017 to 6/2018 at a quaternary care children's hospital. Students were randomized to TDS or SPM during each 2-week block. All students completed a critical appraisal tool (CAT) of evidence related to a clinical question in a standardized appraisal form and self-reflected about the EBM process. The primary outcome was the numeric score of the CAT derived by using the validated Fresno tool. Secondary outcomes of knowledge, attitudes, confidence, and self-reported behaviors related to EBM were measured using validated surveys. Statistical analysis was performed using student's t-test for CAT scores and mixed-model procedure (PROC MIXED), with subject as random effect and time as repeated measure for the secondary outcomes. RESULTS: 127 clerkship students were included. Overall, there was no significant difference in mean CAT scores for TDS (n=59) versus SPM (n=66) groups (90.3 vs. 92.0, p=0.65). There were no significant differences between SPM and TDS groups for knowledge (p=0.66), attitudes (p=0.97), confidence (p=0.55), and accessing evidence (p=0.27). Both groups showed significant gains in knowledge, attitudes, confidence, and accessing evidence from baseline to post course. Improvements in knowledge and confidence were sustained at 3-months. CONCLUSION: A SPM learning module is as effective as a TDS module for application of EBM concepts and knowledge to patient care. To evaluate the effectiveness of a traditional didactic session (TDS) as compared to a self-paced, interactive, multimedia module (SPM) on the application of evidence-based medicine (EBM) skills among medical students during their inpatient pediatric rotation. We conducted a randomized controlled trial from 6/2017 to 6/2018 at a quaternary care children's hospital. Students were randomized to TDS or SPM during each 2-week block. All students completed a critical appraisal tool (CAT) of evidence related to a clinical question in a standardized appraisal form and self-reflected about the EBM process. The primary outcome was the numeric score of the CAT derived by using the validated Fresno tool. Secondary outcomes of knowledge, attitudes, confidence, and self-reported behaviors related to EBM were measured using validated surveys. Statistical analysis was performed using student's t-test for CAT scores and mixed-model procedure (PROC MIXED), with subject as random effect and time as repeated measure for the secondary outcomes. 127 clerkship students were included. Overall, there was no significant difference in mean CAT scores for TDS (n=59) versus SPM (n=66) groups (90.3 vs. 92.0, p=0.65). There were no significant differences between SPM and TDS groups for knowledge (p=0.66), attitudes (p=0.97), confidence (p=0.55), and accessing evidence (p=0. 27) . Both groups showed significant gains in knowledge, attitudes, confidence, and accessing evidence from baseline to post course. Improvements in knowledge and confidence were sustained at 3-months. A SPM learning module is as effective as a TDS module for application of EBM concepts and knowledge to patient care. A self-paced interactive online module is as effective as a traditional didactic session for teaching evidence-based medicine to pediatric clerkship students. No significant differences existed in the critically appraised topic score. Both modalities resulted in improved attitudes, confidence and behaviors. In 2014, the Association of American Medical Colleges created Core Entrustable Professional Activities (EPAs), which serve as a framework for activities that graduating medical students (MS) should be able to perform at the start of residency. EPA 7 is focused on evidence-based medicine (EBM) in which MS are expected to form clinical questions and retrieve evidence to advance patient care. 1 The inclusion of the EPA 7 underscores the need for effective EBM curricula for MS to help them prepare for careers as practicing physicians. Several studies have evaluated the effectiveness of teaching EBM to undergraduate MS utilizing various methods, but there is no standardized approach to teach EBM in the undergraduate medical setting. Prior methods of EBM teaching have been quite heterogeneous and included problem-based learning, 2 peer-assisted learning, 3 lectures, mini-courses, 4 seminars, or selfinstructed learning. 5, 6 The outcome measures and the results in these studies have varied greatly, further emphasizing a lack in standardization in teaching EBM to undergraduate MS. Studies report that students are more likely to adopt EBM principles when there is a demonstrable link between classroom teaching and clinical application. 7, 8 For this reason, integration of EBM teaching is important during clerkships when students engage in patient care. Historically, traditional didactic sessions (TDS) have been a mainstay method of teaching, 5, 6, 9, 10 but TDS may be difficult to sustain during clerkships due to time constraints of students and limited availability of instructors. Furthermore, amid the COVID-19 pandemic, there has also been a rapid shift to promote asynchronous learning through electronic platforms. [11] [12] [13] Electronic self-paced modules (SPM) may be a promising alternative given the flexibility it offers students and the appeal it has to the millennial generation. [14] [15] [16] Studies have compared TDS to SPM in teaching EBM to undergraduate MS, and have found SPM as effective as TDS in improving knowledge and attitudes. 5, 17, 18 However, there is limited data to investigate the effectiveness of SPM for enhancing the application of EBM knowledge and skills to actual patients in the clinical setting. 6, 8, 19, 20, 21 We aimed to compare the effectiveness of a single traditional classroom session (TDS) versus an interactive self-paced, multimedia module (SPM) on the application of EBM skills to an actual patient encounter among pediatric clerkship students. As a secondary aim, we compared changes in knowledge, attitudes, confidence and ability to access evidence between TDS and SPM teaching methods and describe student reflections regarding EBM. We performed a cluster randomized controlled trial at a large, academic, quaternary care children's hospital in Houston, Texas from 6/2017-6/2018. The children's hospital is affiliated with Baylor College of Medicine, which has a four-year medical degree program. MS complete their pre-clinical training during the first three semesters. During the second-half of their second year, students begin their clinical rotations and finish their core rotations by December of their fourth year. Additionally, during the second half of their 2 nd year, students attend an EBM course comprised of nine in-person weekly 1.5 hour sessions occurring at the medical school. Sessions are taught using team-based learning and cover fundamental EBM concepts including PICO (Patient, Intervention, Comparison, Outcome) questions, literature search, and appraisal of different study types. Within the six week Pediatric core rotation, two weeks are spent in the inpatient setting on the Pediatric Hospital Medicine (PHM) service. The PHM team is comprised of one teaching attending, a senior supervising resident, 2-3 interns, 2-3 clerkship students and during various parts of the year, a PHM fellow and/or sub-intern. Second, third, and fourth year students on their core Pediatrics rotation were enrolled in this study after providing consent. Sub-interns were excluded from the study. The curriculum was mandatory for students to complete, however participation in the study surveys was optional. Results from the study were independent from students' grades. The study was approved by the local Institutional Review Board. The content of the TDS and SPM curriculum for the two groups was identical. The goals, objectives and content were finalized after review of existing EBM curricula and discussion with stakeholders, which included MS, local EBM experts, faculty and departmental educational leadership. The content of the curriculum reviewed basic EBM principles, including forming a PICO question, tools to perform efficient literature searches, study design, and critical appraisal basics. (Supplemental Document A) Additionally, all students received resources outlining ways to formulate a PICO question, how to perform an efficient literature search, relevant EBM formulas, and a Critically Appraised Topic (CAT) form. The students randomized to TDS received teaching during the first day of their two-week PHM block. The TDS included an interactive, in-person teaching session facilitated by faculty involved in this study. PHM faculty that participated in the division's EBM interest group volunteered to facilitate teaching sessions and underwent a training session led by an experienced instructor of EBM (SW). Student attendance was tracked with a sign-in sheet. A facilitator's guide was created to orient instructors to the content and format of the session. TDS was approximately 60 minutes in length and consisted of a power point presentation with a case example to illustrate the basic concepts of EBM and its application to clinical practice. Various discussion points were embedded in the lecture to enhance interaction, assess student knowledge, and clarify EBM concepts. The SPM module was an interactive, multimedia module with voice over narration. It was created using Microsoft PowerPoint and Camtasia screen recorder and video editor, covering the same topics in the TDS. It took approximately 60 hours to develop. Voiceover narration, video clips, and interactive short quizzes were incorporated throughout the module, which was exported as an interactive web page module. Incorrectly answered quiz questions prompted the module to go back to the relevant portion to review the concepts again before proceeding further. Students randomized to the SPM were asked to complete the module during the first week of the rotation. Students received emails at set times during the rotation (day 1 and 7) with reminders to complete the module. During the second week of their PHM rotation, students in both groups were asked to formulate a clinical question based on a patient they cared for during the rotation. Students were expected to complete a CAT form based on the patient encounter. The form prompted students to create a PICO question, identify an ideal study design for their question, write search terms, select an article, and perform a basic appraisal of the article. (Supplemental Document B) The students turned in the CAT form at the end of their rotation along with their history and physical exam note for the patient. To align with EPA 7, students were asked to write a reflection statement at the end of the CAT form asking what practicing EBM meant to them after completing the exercise. We created a computer-generated randomization scheme, where each two-week block of students were randomized to receive either TDS or SPM. The investigator (SK) created the scheme and was unaware of student rotation assignments. The rotation schedules for students were made by individuals outside of the study team from the Undergraduate Medical Education office, whom were unaware of the randomization schedule. Students were notified of their educational intervention at the start of their rotation. To avoid contamination between groups, only students assigned to the SPM were given instructions on how to access the learning module. Due to the nature of the intervention, we were unable to blind students and TDS facilitators. Blinding was utilized for assessment of the primary outcome. A single evaluator (AD), blinded to the group assignment and student names, graded the forms. AD did not facilitate any teaching sessions nor work on the inpatient service during the study period. The primary outcome was a numeric score given to the CAT forms based on the previously validated Fresno tool. 22 (Supplemental Document B) The Fresno tool was modified to include the domains taught within the session and to stay in-line with EPA 7. The modified Fresno tool evaluated six domains on the CAT form (PICO question, study design, search terms, validity, magnitude/statistical significance, and relevance). (Supplemental Document C) Each section ranged from a maximum score of 16-24 points, and overall maximum score was 124. A single investigator (AD) scored the CAT forms. Prior to beginning scoring, AD participated in a training session with the primary investigator (TH) for which they reviewed four sample CATs together. A posthoc analysis for the reliability of the scoring was performed by independent review of 10% of the CATs by a second investigator (SW) and showed good inter-rater reliability (intraclass correlation coefficient 0.77). SW was blinded to the original scores. Secondary outcomes were measured via a survey administered to the students before the educational intervention, at the end of the two-week block, and 3 months post educational intervention. The survey questions focused on knowledge, attitudes, confidence, and accessing evidence of various data sources. The survey was modeled after a previously validated KACE questionnaire. 23 (Supplemental Document C) The KACE questionnaire was originally developed for dental students, so minor modifications were performed to fit our study population. 23 To assess the frequency of accessing evidence, questions from the "Critical Appraisal Skills Programme Workshop Evaluation Questionnaire" were also added to the survey. 24 The survey was piloted with faculty within the study team and also cognitive interviews were done with three MS prior to the start of the study to ensure clarity and understandability of the questions. Descriptive statistics were used to describe the characteristics of students in the TDS and SPM groups. Characteristics were compared as per the intervention received using Pearson's chi squared test and Fisher's exact test. The primary outcome of mean CAT score in each group was compared using the student's t-test. Prior to using the t-test, Levene's test was used to determine the homogeneity of variance and met the assumption (p=.55). The mean CAT scores were also compared in the subgroup of MS-2 students that had not yet taken the EBM course at the medical school using the Wilcoxon Rank Sum test. The secondary outcomes i.e. the average scores on knowledge, as well as 5-point Likert scale items for attitudes, confidence, and frequency of accessed data sources were analyzed using repeated-measures analysis using the mixed model procedures (PROC MIXED with REPEATED statement to take into account the covariance structure of the data). The protocol defined in PROC MIXED used the REML (restricted maximum likelihood) option with subjects as random effect and time as repeated measure with an autoregressive heterogeneous (ARh) covariance matrix option. The ARh covariance matrix was used because the time period between sessions varied in the protocol. The least square means (LS means) statement of SAS PROC MIXED was used and compared between sessions and also sessions at each time point using the Student's t test. Missing data was assumed missing at random. To assess for attrition bias due to missing data at 3 months, a sensitivity analysis was performed with inclusion of only students that completed all three time points of data for secondary outcomes. A p-level of ≤.05 was considered significant. Data analysis was performed using SAS version 9.4 (SAS Institute, North Carolina). A summative content analysis was performed to describe reflection statements and quantify categories of content based on key words. 25 Two investigators (TH and SW) reviewed reflection statements to develop a code book of categories. TH quantified the proportion of reflections that contained key words within categories. Data was organized using Excel. 127 of 130 students consented and enrolled in the study. (Figure 1 ) Sixty four students were randomized to the TDS group and 63 students to SPM. One session with four students was reassigned from TDS to SPM due to the lack of instructor availability during the week of a national holiday. There was 100% attendance for students randomized to TDS. Of the students enrolled, 49% were MS2s, 41% were MS3s, and 10% were MS4s. The characteristics of students in TDS and SPM groups were similar except for field of interest, where TDS had a larger percentage of students with multiple interests or undecided and SPM had a larger percentage of students with interest in internal medicine. (Table 1) CAT forms were completed in 125 of the 127 students enrolled; students reported spending an average of 90 minutes to complete it. There was no significant difference in mean CAT scores between the TDS and the SPM sessions Figure 2 ) Similarly, attitudes toward EBM had a small but statistically significant improvement overall but there was a slight decline in attitudes between post-intervention and 3 months. There were no significant differences in knowledge, attitudes, confidence, and accessing evidence between TDS and SPM groups across time points. (Table 2, Supplemental Table 1 ) UpToDate and colleagues were the most commonly used resources, while meta-search engines were used least frequently. Due to low 3 month post-course survey completion, a sensitivity analysis was performed on the 41/127 (32.2%) students that completed surveys at all time points, pre-, post-, and 3-months post. The results were similar to the full population with improvement in knowledge, attitudes, confidence, and accessing evidence between pre and post course time points in each group. (Supplemental Figure 1 ) Interestingly, improvements in confidence and accessing evidence post-course were greater in TDS compared to SPM (p=0.03, p=0.05, respectively). (Supplemental Figure 1 ) Confidence was sustained at 3 months in TDS and SPM. However, just as with the full population accessing evidence was not sustained at 3 months and attitudes declined. (Supplemental Figure 1) To keep in-line with EPA7, students provided their perspectives on the EBM process in reflection statements. Summative content analysis was performed on 115 reflection statements and 165 key words were created. Six categories emerged: utility of EBM (n=49, 30%), goal setting (n=45, 28%), efficiency of integrating EBM in practice (n=30, 18%), importance of EBM (n=20, 12%), clinical application (n=14, 9%), and the usefulness of the assignment (n=7, 4%). Our study is unique in that we compared two different teaching modalities via randomization to identify the most effective way to teach students how to apply EBM principles to real patient cases. We found no significant difference in CAT scores between the TDS and SPM groups, which suggests that SPM is as effective in teaching and applying EBM principles as the TDS in students that have had previous exposure to EBM. This finding is important for educators who aim to integrate EBM teaching in the clinical setting. In our setting, the TDS and SPM sessions followed by the CAT assignment served as review of fundamental EBM concepts plus application of the concepts to a real patient case carried by the student. We were pleased to see that the students' general reflections encompassed not only the importance of EBM but strategies for efficient integration into practice, the utility of EBM and even specific future goals. The reflections show how teaching application of EBM in the clinical setting puts EBM knowledge into meaningful context for the students. When determining which modality of teaching is best suited for a program, educators should weigh the challenges of developing SPM with the challenges in sustaining TDS. TDS requires availability of instructors to sustain this mode of teaching whereas SPM comes with the cost of the software, skills needed to use the software and time. For the secondary outcomes, our results showed small but statistically significant gains in knowledge, attitudes, confidence, and behaviors accessing evidence from pre-to immediately post-course with SPM and TDS. Knowledge and confidence gains were sustained at 3 months post-course in SPM and TDS. However, there was a lack of improvement for accessing evidence and mild decline in the attitudes toward EBM after 3 months. This suggests that a teaching session in conjunction with application of principles during clerkship can reinforce concepts and make students more comfortable applying EBM in the clinical setting, but more longitudinal teaching may be necessary to change behaviors and attitudes towards EBM over time. Several studies exist of EBM curricula implemented during clerkship years with similar results. Modalities of teaching varied from a seminar-based series to on-line education in these clerkship curricula. [26] [27] [28] [29] [30] [31] These studies demonstrated similar results to our study with increased knowledge, attitudes, usage of clinical evidence, and improved critical appraisal skills. [27] [28] [29] [30] Fewer studies exist to examine on-line methods of teaching clerkship students fundamental EBM skills. 26, 31 One study by Schilling, et al. implemented an online curriculum during a family medicine clerkship rotation and compared EBM skills in students that received the online modules to those that did not. The intervention group reported improved confidence in information retrieval, and overall had higher quality search strategies and information retrieval compared to the control group. 31 Aronoff et al. utilized a novel longitudinal approach for on-line learning over 18 weeks. Competency in EBM was measured via Fresno score and showed overall significant improvement in scores post course. Outcomes related to confidence, attitudes and behaviors were not reported. 26 In our study, we directly compared traditional and on-line methods of teaching, which were studied separately in these prior studies and found no difference in how well students applied EBM skills to real patient cases. Gains in knowledge, confidence, attitudes and behaviors were also similar across both groups, which further supports use of either SPM or TDS for teaching EBM. The longitudinal approach to on-line learning utilized by Aronoff et al. seems to be a promising method to help sustain positive attitudes and behaviors throughout the clerkship, as they assimilate EBM skills into their practice. The need for longitudinal curricula across medical school clerkships is supported by our findings 3 months after our intervention. Students reported a mild decline in attitudes toward EBM and a decline in the frequency of accessing evidence. By including EBM curricula throughout clerkships, students could continue to practice strategies to identify evidence relevant to their clinical questions and become more comfortable with the uncertainty that can arise when only small studies or lower quality studies are available. Furthermore, suboptimal role models for evidence-based practice are a barrier to teaching. 32 Certainly, this barrier would be more pronounced when trying to implement longitudinal EBM curricula. The SPM mode for asynchronous teaching and utilization of academic health librarians as instructors could be tools to help overcome this barrier. 33 Integrating a librarian led session to teach and reinforce efficient search strategies related to clinical questions could have a sustainable impact on students. Institutions with limited resources could seek faculty with EBM expertise from other institutions and hold virtual webinars or identify interested faculty to train in EBM instruction through courses such as those offered through the Centre of Evidence-Based Medicine. 34 Further studies are needed to evaluate effective methods to teach EBM longitudinally, which are sustainable, and focus on the application of EBM principles that engage millennial learners. [14] [15] [16] Lastly, due to COVID-19 pandemic, educators have had to create innovative educational opportunities and there has been a quick shift towards virtual learning. The shift to distanced learning may serve as a footing for continued virtual teaching options even in the post-pandemic era. [11] [12] [13] Since completion of this study, the SPM has been incorporated into the clerkship curriculum and feedback has been positive. We have several strengths and limitations to our study. One strength is the design of our study with randomization and blinding. Our study, however, utilizes a single center with students from one medical college and these students do receive one EBM course during their 2 nd year of medical school. It is uncertain whether the results would differ for MS who do not receive any early education in EBM. It is possible that one mode of learning may be more effective over the other for these students. We had a small sample of MS-2 students (n=11) that did not complete the course at the medical school prior to participation and their mean scores were lower. Further studies are needed to evaluate the most effective mode of teaching for students that receive EBM education for the first time during clerkship. Additionally, we were limited to a one year time frame for the study before adoption of SPM formally as part of the curriculum. We did not have literature available from prior research to guide a sample size calculation but our estimates of the primary outcome, the CAT scores, are very similar between groups with overlapping, narrow confidence intervals making type II error unlikely. Another limitation of this study is the low three-month response rate for the survey results. We are reassured, however, that outcomes measured in students that completed all 3 times points of surveys were similar to the full population making the likelihood of attrition bias low. Additionally, we conducted analyses as per the modality of teaching received rather than the group assigned but there was minimal cross-over between groups and the characteristics of students in SPM and TDS remained similar, making bias unlikely. Blinding of facilitators or students was not possible due to the nature of the interventions but the students were unaware of the grading rubric used for the CATs and the grader for the CATs was blinded to group assignment and student identity. Lastly, in students randomized to the SPM, adherence to completion of the module, the timing of viewing the module during the rotation and time spent on the module was not formally tracked. SPM is as effective in teaching and applying EBM principles as TDS during medical school clerkship rotations. Institutions should balance the availability of resources to determine the preferable mode of teaching at individual programs. Further studies should address whether longitudinal EBM curricula through clinical clerkships can be used to sustain favorable attitudes and behaviors towards evidence-based practice. For modules at each time point: Change * = Time 2 -Time 1; A positive change indicates improvement and negative change indicates worsening of the outcome; Pre to 3 month post comparison was used to assess sustainment of outcomes and post to 3 month post comparison was used to assess any decline in the outcome. p-values < 0.05 are bolded -"EBM is a vital part of practicing medicine and is becoming more and more relevant to our patients. As a result, it is important to include EBM in daily practice." -"EBM should be the cornerstone of any practicing physician today." -incorporate -plan to use/keep up -will ask questions -intend to read/use/find answers/practice -will read -will ask questions -look at guidelines -set a goal -need to improve -will critically appraise -build a habit -continue to apply -"I will read at least a few articles per week to keep up with the latest advances in the field." -"It will take more practice to effectively interpret such studies and trials, but my hope is that the more I incorporate into my own practice, the easier and more meaningful it will become." -"Efficient ways to access evidencebased medicine include Cochrane reviews, hospital-specific clinical guidelines and algorithms, metaanalyses, and systematic reviews. Another good resource is the RCT, especially if the magnitude of effect and quality of evidence is significant enough to warrant a change of guidelines. I intend to use the above mentioned resources in my practice." -management -diagnostic testing -diagnosis -counseling -risk factors management -imaging -antibiotic usage -"Given the results of this study, I will carefully consider the adverse side effects and potential consequences of obtaining CT imaging in pediatric patients." -great learning tool -valuable -helpful -will benefit me -useful -"The skills that I have learned through this process will benefit me in the future as I can better evaluate sources I find." -"This exercise was helpful because it make me think about a patient who wasn't my own and allowed me to really feel like I was a part of her care." A randomized-controlled trial of two educational modes for undergraduate evidence-based medicine learning in Asia Evaluation of Peer Assisted Learning in Evidence Based Medicine Course: A Pilot Study at University of Glascow Successful teaching in evidence-based medicine Self-directed Learning can Outperform Direct Instruction in the Course of a Modern German Medical Curriculum -results of a mixed methods trial Effectiveness of Teaching Evidence-based Medicine to Undergraduate Medical Students: a BEME Systematic Review Undergraduate Medical Student perceptions and use of EBM: A qualitative study Teaching Evidence Based Medicine should be Integrated into Current Clinical Scenarios Educational Strategies for Teaching Evidence-based Practice to Undergraduate Health Students: Systematic Review Integration of Evidence-based Practice in Bedside Teaching Paediatrics Supported by E-Learning What You Gain and What you Lose in COVID-19: Perception of Students on their Education Coalition for medical education -a call to action: a proposition to adapt clinical medical education to meet the needs of students and other healthcare learners during COVID-19 The impact of COVID-19 on the undergraduate medical curriculum Twelve tips for facilitating Millennials' Learning Anatomy of Education for the YouTube Generation Milestones and Millennials: A Perfect Pairing: Competency-Based Medical Education and the Learning Preferences of Generation Y Comparison of directed and self-directed learning in evidence-based medicine: a randomized controlled trial Computer-based teaching is as good as face to face lecture-based teaching of evidence based medicine: a randomized controlled trial What is the evidence that postgraduate teaching in evidence based medicine changes anything? A systematic review Internet-Based Learning in the Health Professions: A Meta-analysis A Hierarchy of Effective Teaching and Learning to Acquire Competence in Evidence-based Medicine Validation of the Fresno Test of Competence in Evidence Based Medicine Validation of an Instrument to Assess Evidence-Based Practice (EBP) Knowledge, Attitudes, Access, and Confidence Development and Validation of a Questionnaire to Evaluate the Effectiveness of Evidence-based Practice Teaching Three Approaches to Qualitative Content Analysis. Qualitative Health Research Integrating Evidence Based Medicine into Undergraduate Medical Education: Combining Online Instruction with Clinical Clerkships Impact of an evidence based medicine curriculum on medical students' attitudes and skills Changes in knowledge and skills of Year 3 undergraduates in evidence-based medicine seminars Evidence-based medicine curriculum: impact on Medical students Teaching critical appraisal: a pilot randomized controlled outcomes trial in undergraduate osteopathic medical education An Interactive Web-based Curriculum on Evidence-based Medicine: Design and Effectiveness Challenges to Learning Evidence-Based Medicine and Educational Approaches to Meet These Challenges: A Qualitative Study of Selected EBM Curricula in US and Canadian Medical schools Entrustable professional activity 7: opportunities to collaborate on evidence-based medicine teaching and assessment of medical students