key: cord-0990437-woodpmfx authors: Grech, Marco; Grech, Stefania title: An Analysis of the Educational Environment at the Malta Foundation Programme Using the Postgraduate Hospital Educational Environment Measure (PHEEM) date: 2021-07-23 journal: J Med Educ Curric Dev DOI: 10.1177/23821205211035640 sha: 99a2042d4ffa47343ec53f879618a216c2f5c4e8 doc_id: 990437 cord_uid: woodpmfx BACKGROUND: Learning from managing patients in a real-world context is by far superior to simulation learning. A substandard educational environment is associated with poor patient care and suboptimal learning outcomes. The measurement of the educational environment provides insight into what is needed to improve the level of training. OBJECTIVE: To measure the educational environment as perceived by trainees within the Malta Foundation Programme. METHODS: This study used the Postgraduate Hospital Educational Environment Measure (PHEEM) to measure the educational environment at the Malta Foundation Programme. Descriptive statistics were used to describe the demographics of the study population. Nonparametric comparative statistics were used to identify statistically significant differences between groups. RESULTS: Ninety-eight trainees out of 370 (26.5%) completed the online questionnaire. These consisted of 39 FY1s (31.5% of 124), 33 FY2s (24.8% of 133), and 26 extended FYs (23.0% of 113). The 40-item PHEEM showed good reliability with a Cronbach’s α value of .912. These doctors perceived their educational environment as more positive than negative. Perceptions are worst among trainees at the end of their first year of training. Those who had just finished their training have reported better perceptions. The 3 worst scoring items are related to when the trainees are on call. CONCLUSION: Among trainees within the Malta Foundation Programme, perceptions of role autonomy and social support are areas where most work is needed. Teaching seems to be moving in the right direction, but there is always room for improvement. For healthcare professionals, there is no substitute for learning in the clinical environment. 1 Learning from managing patients in a real-world context is by far superior to simulation learning. Over the past 3 decades, the subject of the educational environment, both at the undergraduate and postgraduate levels, has been the focus of much research and discussion among all stakeholders in medical education. Both the General Medical Council (GMC) 2 and Kilty et al 3 have underlined that a substandard educational environment is associated with poor patient care and suboptimal learning outcomes. Not only do young trainees learn from their work with patients, but patient care would suffer were these trainees to be removed from teaching organizations as their work is essential for any healthcare organization. This is further confirmed by the GMC 2 when it states, "patient safety is inseparable from a good learning environment and culture that values and supports learners and educators." The educational environment is, therefore, considered an important measure in both undergraduate and postgraduate medical training. Postgraduate medical trainees have a dual contrasting role. On the one hand, they are trainees transitioning to hospital life, still in need of support and still adapting to their new responsibilities. 4 On the other hand, as practicing doctors, they need to provide answers to patients' questions, work long hours, be accountable for their actions and may find inadequate support from their more senior colleagues. Trainees may feel under constant evaluation by their patients, colleagues, and senior grades. 5 This dual role is known to increase the risk of stress, anxiety, depression, and burnout. 6 Medical trainees are known to be one of the most vulnerable categories of employees to occupational stress. 7 The MACY Foundation 8 sees the educational environment as: "the social interactions, organizational cultures and structures, and physical and virtual spaces that surround and shape participants' experiences, perceptions, and learning." 8 The American Medical Association defines the educational environment as: "a social system that includes the learner, the individuals with whom the learner interacts, the setting(s) and purpose(s) of the These definitions, among others, identify the complexity of the educational environment when compared to education in a class room. The measurement of the educational environment provides insight into what is needed to improve the level of training. This is done by identifying the weaknesses and strengths of a training program and which areas should be prioritized in any improvement that is planned. For many years, the educational environment was ignored, possibly because of the lack of suitable and validated instruments that could be used to assess it. This has now changed, and a number of validated instruments have been used to assess the different aspects or domains of the educational environment. One of the first to be developed was the Dundee Ready Education Environment Measure (DREEM). 10 DREEM uses 5 subscales in its evaluation of the education environment: • Students' perception of learning • Students' perception of teachers • Students' academic self-perceptions • Students' perceptions of atmosphere • Students' social self-perceptions Following DREEM, a number of other instruments aimed at assessing the education environment in different settings were developed and validated. List 1 provides a selection of instruments for evaluating specific learning environments in medical education. PHEEM was specifically created to measure the educational environment within a postgraduate setting. Thus, PHEEM was an ideal tool for this study. With the use of different measures of the education environment now widespread in the literature, such instruments can be used to gain holistic views of the curriculum; understand the students' perceptions of learning, teaching, and the atmosphere; compare the different perceptions of the various stakeholders; compare environments between schools or departments; provide organizations with indications of what needs to be changed or improved; and evaluate the results of any changes made to the education environment. 29 To measure the educational environment as perceived by trainees within the Malta Foundation Programme. The aim of this study was to analyze the educational environment within the Malta Foundation Programme as perceived by the trainees within the program themselves. The Malta Foundation Programme (MFP) 30 was founded in 2009. The Malta Foundation Programme is a 2-year training program for newly graduated doctors consisting of structured teaching, hands-on training, and assessments while working in a supervised hospital or primary-care environment. As the MFP bases its operations on the same Reference Guide while offering the same Curriculum and training opportunities as the UK Foundation Programme (UKFPO), the MFP was awarded the status of Affiliate Program by the UK Foundation Programme. This status has ensured that trainees of the MFP, upon completion of their programme, can compete on the same level as those completing the UK Foundation Programme for training posts in specialities in Malta, the UK, or elsewhere. This affiliation has since been renewed at regular intervals after the MFP repeatedly fulfilled the Quality Standards of the Malta Medical Council and the UKFPO. 30 A cross-sectional observational method was chosen for a study of the educational environment and burnout among foundation doctors in Malta. It is the author's personal opinion that the educational environment at the Malta Foundation Programme may be related to possible burnout among the trainees. The aim of the study was to analyze both the educational environment and burnout at the Malta Foundation Programme. This paper reports on the results for the educational environment. The other results are reported elsewhere. The instrument chosen to assess the educational environment is the PHEEM questionnaire administered online through Google Forms to all doctors within the Malta Foundation Programme. The PHEEM questionnaire has been validated in various settings and in different countries. Minor changes were made to the wording of the questionnaire to ensure that each question was relevant to the Maltese setting. Direct correspondence with the original author of the PHEEM questionnaire ensured permission to use and validity of the minor changes to the wording. PHEEM was administered in English, as all medical tuitions in Malta are carried out in English. The questionnaires were distributed to all foundation program doctors via their year representatives in the form of a Google Form online questionnaire. As the Foundation Programme does not offer the service of forwarding emails directly, the questionnaire was initially sent to the Foundation School Secretary, who then passed it on to the foundation doctors' representatives. A reminder was sent after 7 days to increase the response rate. All respondents were requested to submit solely a single reply, especially as this was an anonymous questionnaire. The period of data collection coincided with a period when 3 cohorts of foundation doctors could be sampled. The first years were sampled at the end of their first rotation, 3 months into practice. The second years were sampled between their first and second years, a year into practice. A third category, though technically not under the responsibility of the Foundation Programme, was the extended Foundation doctors. These were doctors who had successfully finished their foundation training and were waiting for their BST posts to be decided. These trainees were sampled at the end of their 2-year foundation training. The sampling period ran between July and August 2020 for FY2s and extended FYs and in October 2020 for FY1s.The deadline for the collection of data was 2 weeks after the reminder was sent. The responses were analyzed using SPSS 25.0. Questions 7, 8, 11, and 13 are negatively worded statements and need to be reversed for scoring. Reliability tests of the whole instruments and the 3 factors were performed using Cronbach's alpha. Descriptive statistics were used to describe the demographic features of the respondents. Tests for normality (Kolmogorov-Smirnov and Shapiro-Wilk) for all factors revealed that no question was normally distributed. Therefore, all comparative statistics were of the nonparametric type. The Mann-Whitney U test was used to analyze differences between sexes. The Kruskal-Wallis H test was used to test for differences between foundation years. The threshold for statistical significance was a P < .05 and 95% confidence interval. Factor analysis of all 40 items was also performed using both the scee plot and a criterion of an eigenvalue >1. Ethical approval was obtained from the University of Malta Faculty Research Ethics Committee, and approval to disseminate the questionnaire was obtained from the Malta Foundation Programme. A covering letter and a participant information sheet accompanied the questionnaire. Informed consent was obtained from all participants. All methods were carried out in accordance with relevant guidelines and regulations. The 40-item PHEEM showed good reliability with a Cronbach's α value of .912. Cronbach's α for each of the 3 subscales of the PHEEM was as follows: Autonomy-.790 Teaching-.885 Social support-.683 The responses of each FY cohort to each of the 40 items forming up the PHEEM are listed in Table 2 . Table 3 illustrates the cumulative scores for the 3 themes and for the total PHEEM score for each of the 3 cohorts assessed in this study. The authors of PHEEM recommend that any item with a mean of 2 or less should be examined in detail as it may indicate a problem area. Table 4 provides a summary of the items with a mean total score below 2 for the 3 cohorts in the study. The items have been arranged in ascending order. There was only 1 question that was rated >3 by the 3 cohorts: "I have good collaboration with other doctors in my grade." Another question, "My clinical teachers promote an atmosphere of mutual respect," was rated 3.00 by Foundation Year 1 respondents. The original authors of PHEEM have suggested that items with a mean score of 3.5 or over are real positive points. None were identified in this study. Statistically significant differences between genders were identified on 7 items, as listed in Table 5 . In general, females seemed to give higher rankings in almost all 7 items. The only exception is the question regarding sex discrimination in the workplace. Table 6 reports on the mean total PHEEM and perceptions of autonomy, teaching, and social support by gender. Statistically significant differences between foundation-year cohorts were identified in 5 items using the Kruskal-Wallis H test. These results are illustrated in Table 7 . Generally speaking, mean scores seem to dip midway through the 2-year training period, only to pick up again at the end of training, but never reaching the same levels as at the early stages of training. The current study evaluates foundational doctors' perceptions of the educational environment at the Malta Foundation Programme. These doctors perceived their educational environment as more positive than negative. The educational environment scores worst among trainees at the end of their first year of training. Those who had just finished their training have reported better perceptions but these do not reach the levels of those at the start of training. This dip could signify that the perception of the educational environment hits a low midway through the 2-year period, but as their career progresses, foundation doctors are quick to adapt. As a result, their perception of the educational environment improves toward the end of the training program. One needs to keep in mind that these results are the reflection of the perception of different cohorts. A longitudinal observational study can provide more concrete results to explain this dip. Perceptions of teaching and social support ranked higher than perceptions of role autonomy. Trainees have demon- strated particularly low rankings in perceptions of role autonomy midway and at the end of training. Interestingly, out of the 11 items with a mean score less than 2, 6 form part of the autonomy subscale, 5 form part of the social support subscale, and only 2 were part of the teaching subscale. The 3 worst scoring items seem to be related to when the trainees are on call. Malta persists with a system of on call Grech and Grech 7 duties that has been scrapped in many countries. Under the current conditions, foundation doctors work for some 28 hours at a stretch. While adoption of the European Working Time Directive is part of Maltese law, foundation doctors may feel that if they choose not to exceed 48 hours of work in a week, they may find themselves at a disadvantage when applying for training posts at a later stage in their career. On call duties are made worse by the lack of proper catering facilities on site and by an inordinate number of inappropriate pages. Statistically significant differences between genders were identified on 7 items, as illustrated in Table 4 . Five of these items form part of the perceptions of role autonomy subscale, and 2 form part of the perceptions of social support subscale. Female trainees gave higher mean rankings for all 7 items except for "There is sex discrimination in this post." It is encouraging to note that female trainees do not perceive the educational environment at the Malta Foundation Programme to be sexually discriminatory. Foundation Year 1 trainees scored significantly higher on 5 items, as shown in Table 5 (3 items on the autonomy subscale and 2 on the social support subscale). There seems to be a trend of deterioration in perceptions across all items and subscales as training reaches its midpoint. Perceptions then improve by the end of training but never reach the same levels of ranking achieved at the start of training. Whether this could be related to the development of burnout in trainees will be analyzed in a further study. As this is an observational, not longitudinal, study conclusions are guarded in this respect. In a similar local study, Farrugia Jones and Cacciotolo 31 assessed the postgraduate educational environment in the Department of Medicine at Mater Dei Hospital, Malta. The response rate for house officers in this study was 10%. Problem areas identified by trainees in this study were the absence of protected time, no access to an individual educational program and working hours that do not conform with the European Working Time Directive. The same problems have, unfortunately, again been highlighted by this study. Poor catering facilities, poor access to career advice and counseling opportunities, a strong blame culture, and an inappropriate workload were also replicated in the current study. There are multiple benefits of a healthy educational environment. Studies have shown that the learning environment is associated with the quality of care provided. 32 The educational environment also influences the prescribing habits of trainees 33 and the management and use of health care services and resources. 34 In surgical training, higher rated surgical training programs were associated with lower complication rates. 35 A healthy learning environment has been associated with better residents' outcomes including satisfaction with training, the use of knowledge, and a professional identity development. 36, 37 The importance of having a high-quality clinical learning environment stems from the knowledge that a high-quality learning environment will have a direct impact on workplace learning and, on the quality, and safety of the clinical care received by patients. Trainees exposed to a high-quality clinical learning environment will have been exposed to participation in the clinical care of patients in a supervised manner. They will also be exposed to coaching, assessment and feedback, deliberate practice, and peer collaboration. 3 Table 4 . Summary of the items with a mean total score below 2 for the 3 cohorts in the study. The items have been arranged in ascending order. Many trainees are meant to learn and develop their clinical skills in understaffed, underfunded, uncontrolled, and overcrowded clinical educational environments. 38, 39 Exceeding a certain critical level of workload will result in a decline in trainee learning. Heavy workloads were found to be linked to a number of undesirable outcomes in trainees. These outcomes include an increased likelihood of burnout and lower engagement, health, and well-being. 40, 41 Lower levels of patient satisfaction, poor standards of care, and higher mortality were also associated with self-reported heavy workloads. 42, 43 In view of the above benefits of a healthy educational environment on quality of care, learning, trainees' mental health and safety of care, efforts at improving the role autonomy, and social support aspects of the educational environment need to be prioritized. This study has limitations. The response rate was relatively low despite a reminder being sent. The quantitative nature of the study precludes an in-depth exploration of the reasons why trainees have expressed perceptions. A qualitative additional study is warranted. The study was also carried out in a single center, albeit this is the only 1 on the island. The study did not take into account the effect of a pandemic on the educational environment. This study aimed to evaluate the educational environment of the Malta Foundation Programme. The first 2 years of postgraduate medical education signify a transition that can be rough and demanding, as trainees are suddenly burdened with professional responsibilities. It is a period where learning has not ended with the undergraduate years but is only beginning in the postgraduate years. The current study has identified areas in the educational environment where improvement is needed. Perceptions of role autonomy and social support are areas where most work is needed. Teaching seems to be moving in the right direction, but there is always room for improvement. A comparison with another local study identified similar problem areas. The study was conducted at a time where the COVID-19 pandemic was having its toll on healthcare systems across the world. It may also have had an impact on the results of this study. Dr. Marco Grech is a Senior General Practitioner. He is currently Assessment Lead at the Malta College of Family Doctors and has served as AKT Lead and Education Secretary for the same College in the past. He is a visiting Marco Grech was the main contributor to this manuscript having written and reviewed all the manuscript. Stefania Grech contributed with data collection and analysis and reviewing of the manuscript. All data can be downloaded here: https://doi.org/10.6084/ m9.figshare.13215428.v1. Ethics approval was obtained from the Faculty Research Ethics Committee at the University of Malta. Permission to conduct the study was also obtained from the Foundation Programme, Malta. Informed consent was obtained from all participants. The participants also consented to the publication of results. All methods were carried out in accordance with relevant guidelines and regulations. The clinical learning environment Promoting Excellence: Standards for Medical Education and Training A national stakeholder consensus study of challenges and priorities for clinical learning environments in postgraduate medical education Do burnout and work engagement predict depressive symptoms and life satisfaction? A three-wave seven-year prospective study Burnout in medical residents: a study based on the job demands-resources model Impact of hospital educational environment and occupational stress on burnout among Greek medical residents Occupational stress, job satisfaction and health state in male and female junior hospital doctors in Greece 2018) Improving environments for learning in the health professions Assessing the relationship of learning approaches to workplace climate in clerkship and residency Development and validation of the Dundee Ready Education Environment Measure (DREEM) Selecting instruments for measuring the clinical learning environment of medical education: a 4-domain framework Validation and use of an instrument to measure the learning environment as perceived by medical students Development and validation of the Scan of Postgraduate Educational Environment Domains (SPEED): a brief instrument to assess the educational environment in postgraduate medical education Development and psychometric evaluation of the Undergraduate Clinical Education Environment Measure (UCEEM) Development and validation of the AMEET inventory: an instrument measuring medical faculty members' perceptions of their educational environment The Dundee Ready Educational Environment Measure (DREEM)-a generic instrument for measuring students' perceptions of undergraduate health professions curricula Medical student perceptions of the learning environment at the end of the first year: a 28-medical school collaborative The development of a learning environment: a questionnaire for use in curriculum evaluation The 1960 medical school graduate: his perception of his faculty, peers, and environment The Johns Hopkins Learning Environment Scale: measuring medical students' perceptions of the processes supporting professional formation Development and validation of the Anaesthetic Theatre Educational Environment Measure (ATEEM) Development of an instrument to measure the clinical learning environment in diagnostic radiology Development of an instrument to measure the surgical operating theatre learning environment as perceived by basic surgical trainees Assessing the educational environment in the operating Room-a measure of resident perception at one Canadian institution Development of ACLEEM questionnaire, an instrument measuring residents' educational environment in postgraduate ambulatory setting Development and validation of an instrument to measure the postgraduate clinical learning and teaching educational environment for hospital-based junior doctors in the UK Development and analysis of D-RECT, an instrument measuring residents' learning climate Factors determining medical students' and residents' satisfaction during VA-based training: findings from the VA learners' perceptions survey Essential Skills for a Medical Teacher: An Introduction to Teaching and Learning in Medicine Welcome to the Foundation Programme Postgraduate Health Education Environment at the Department of Medicine at Mater Dei Hospital Effect of a community oriented problem based learning curriculum on quality of primary care delivered by graduates: historical cohort comparison study Predictors of inappropriate antibiotic prescribing among primary care physicians Spending patterns in region of residency training and subsequent expenditures for care provided by practicing physicians for Medicare beneficiaries Using patient outcomes to evaluate general surgery residency program performance Towards healthy learning climates in postgraduate medical education: exploring the role of hospital-wide education committees A narrative review on burnout experienced by medical students and residents Health in Europe 7 financial crisis, austerity, and health in Europe Overcrowding in emergency department: an international issue Stress, satisfaction and burnout among Dutch medical specialists Resident wellness matters Culture and behaviour in the English national health service: overview of lessons from a large multimethod study Patient neglect in healthcare institutions: a systematic review and conceptual model Marco Grech https://orcid.org/0000-0003-2753-6182