key: cord-0838658-cr5reuq1
authors: Heitkamp, Nicholas M.; Snyder, Alana N.; Ramu, Arthi; Shen, Abra H.; Akingbola, Oluwabukola; Malpani, Rohil; Kiiskila, Lindsey; Morgan, Lucas E.; Seeley, Kylie M.L.
title: Lessons Learned: Applicant Equity and the 2020-2021 Virtual Interview Season
date: 2021-09-16
journal: Acad Radiol
DOI: 10.1016/j.acra.2021.08.005
sha: 23e39abe7a29fc8619e333fff39ff4de5742bfc5
doc_id: 838658
cord_uid: cr5reuq1

nan

T he Coronavirus Disease 2019 (COVID-19) pandemic has had a profound impact on medical training in the United States over the past year. In March 2020, the Association of American Medical Colleges (AAMC) temporarily suspended medical student participation on all clinical services nationwide (1) . The National Residency Matching Program (NRMP) and Accreditation Council for Graduate Medical Education (ACGME) followed with a joint statement in May 2020 announcing that the 2020-2021 residency interview season would be conducted virtually (2) . Faced with these challenges, tremendous effort was put forth by all stakeholders to ensure that applicants and programs would be ready for the interview season and the 2020-2021 match.

During this time of residency interview changes, renewed efforts for social and racial equity were sweeping the nationefforts of utmost importance to medical trainees. With a goal of increasing equitable practices À defined as those intending to bring fairness and justice (3) -in the field of medicine, nine medical student co-authors from five different medical specialties, including radiology, collaborated to outline the mechanics of this inaugural virtual interview season with respect to applicant equity. As members of the only class to have participated in a virtual recruitment process, we hope to offer valuable insight for residency leadership regarding how the logistical factors of the 2020-2021 virtual interview season may have affected applicant equity.

Medical students pay a high financial cost for their education; (4) in 2019, the median education debt of medical school graduates was $200,000 USD (5, 6) . In years past, students applying into radiology have spent upwards of $12,000 to attend in-person interviews, with an average total cost of $4.552 throughout the season (7) . During the inaugural "virtual" interview season of 2020, we found the financial savings to be one of the greatest benefits.

Virtual interviewing improved applicant equity by decreasing the financial burden for all students, especially those in financial hardship and with pre-existing debt. Medical students face varying degrees of financial inequity, largely due to the pre-existing socioeconomic division among students (4, 8) . For example, over 44% of U.S. medical students in 2005 reported estimated family incomes of less than $91,705 (9) . Additionally, students who serve as primary caregivers may face added financial burdens. By virtue of travel-related cost savings afforded by the 2020-2021 virtual interview season, students experienced a more equitable process of finding a radiology program with which to train. Some students may have had difficulty obtaining a reliable computer, a stable internet connection, or a professional location from which to interview. The degree to which those underrepresented in medicine (URiM) had more difficulty than others is currently unknown, and it stands to reason that travel-related cost savings could have helped offset some of this disparity.

Decreasing interview-associated costs lowers the overall financial burden of medical school, likely improving applicant equity through increased well-being. Physician mental health and well-being have gained attention in recent years due to the rising awareness of depression, burnout, and suicide among doctors and medical students (10) . A physician's financial stress can be a significant contributor to well-being and is almost certainly impacted by debt incurred during training (11) . In addition, the overall burden of medical school debt is thought to play a role in students' choices of medical specialties. Students with higher levels of anticipated debt are seen opting for higher income specialties to repay those loans, ultimately resulting in fewer students choosing primary care specialties (11) .

The term URiM is defined by the AAMC as "racial and ethnic populations that are underrepresented in the medical profession relative to their numbers in the general population" (12) . Radiology as a specialty and medicine in general lacks diversity in terms of race, ethnicity, and sexual orientation (13) . The physician workforce does not adequately reflect the diversity of the nation's patient populations, a face which ultimately affects the quality of patient care in underrepresented populations (14) .

The advantages of web-based meeting applications, combined with a heightened national movement toward anti-racism and racial equity, might have led to an overall increase in URiM-centered events held by residency programs during the 2020-2021 recruitment season. The decreased physical constrains and increased efficiency of time that the virtual spaces offered improved access to these sessions by applicants from around the country, previously restricted by geography and scheduling during in-person recruitment. The increase in virtual events designed for URiM applicants created a more equitable process.

The knowledge shared during URiM events proved very useful to applicants. Minoritized students have reported feeling less safe and less supported by medical institutions than their counterparts (15) (16) (17) . As the field of medicine works toward adequate inclusion and equity for all, it is important for residency programs to demonstrate the diversity and inclusion efforts being implemented. Due to changes imparted by COVID-19, perspectives of a program's culture and location were harder to ascertain given the pandemic's impact on travel (18) . However, virtual social events devoted to minoritized applicants provided the opportunity to regain such vital information (19, 20) . We noted that the knowledge exchanged from URiM residents regarding their experiences within a particular program provided invaluable information. In fact, some of us made significant rank list decisions directly from information and inclusivity conveyed during such events.

For many applicants, interviewing in their own environment was a convenient experience that likely helped anxiety associated with the interview process. For applicants with disabilities, as well as those with significant home obligations, this switch substantially improved the equity with regard to the residency recruitment process.

Disabled applicants make up a small and likely underreported percentage of medical students in the United States (21) . However, medical students living with disabilities may face additional stress and logistical difficulties during traditional in-person residency interviews due to travel and location accessibility concerns. The advent of the virtual interview season has likely alleviated several concerns for applicants with disabilities. Interviewing from home reduces the physical constraints and stress caused by the travel required for residency interviews. Additionally, virtual platforms give applicants more control over their visual presentation during an interview, allowing them to avoid certain stigmas and biases.

In similar ways, applicants who are also caregivers face challenges with the traditional in-person interview season. For applicants with children, extensive time away could negatively impact their parental duties. For pregnant applicants, in-person interviewing may not provide the option of withholding their pregnancy status and may in fact invite an undue amount of conversation related to the pregnancy. Traveling for in-person interviews is an added stress for breastfeeding mothers and their babies, potentially requiring them to switch to bottle-feeding while away. Thus, for applicants with dependents, virtual interviews decrease time spent away from home and create a more equitable opportunity.

As the residency application process has become more competitive, applicants have applied to an increasing number of programs to improve their chances of receiving an interview (22) (23) (24) . For example, the average number of applications submitted by diagnostic radiology applicants increased from 34.1 in 2016 to 46.5 in 2020 (25) . Applicants are caught in a "prisoner's dilemma," whereby they feel pressure to apply to more programs in an attempt to improve their chances against other applicants who also have increased their number of applications (22-24,26). As there is no limit on the number of applications submitted, over-application represents a major contributing factor to the financial burden of the typical interview cycle (23) . This exacerbates inequities among applicants, as financially advantaged students can more easily afford to apply to additional programs. As programs face an ever increasing number of applications, they often rely more heavily on performance metrics, most notably board scores, to filter applications to a more manageable number (24) . Such widespread use of standardized testing, whose value in predicting future success as a physician is questionable, (27, 28) tends to screen out underrepresented students, for when compared to a white male reference group, Hispanic and Black examinees were found to score 12.1 and 16.6 points lower on USMLE Step 1, respectively (29, 30) .

Many graduate medical education (GME) leaders reported anecdotal evidence of increased applications received during the 2020-2021 "virtual" interview season (31, 32) . This could be due to the fact that applicants did not incur the immense travel costs associated with traditional in-person interviews, allowing them to reallocate their budgeted expenses for additional applications. The burden of over-application is particularly troublesome for less-competitive US medical graduates and international medical graduates (IMGs). IMG applicants face adversity when applying to US residency positions for reasons not discussed in this paper. In order for these applicants to compete within a market saturated by US medical graduates, they often apply to a very large number of schools in hopes of receiving an adequate number of interview invitations (22, 33) . The virtual season magnified the over-application phenomenon leading to further inequities for IMG colleagues.

The modern interview scheduling process has become inequitable for many applicants. Most residency training programs utilize an interview brokering service to schedule their interviews such as ERAS (AAMC, Washington, D.C.), Thalamus (SJ Medconnect, Santa Clara, CA), or Interview Broker (The Tenth Nerve, Los Altos, CA). Employing these services reduces scheduling logistics that would otherwise fall on busy program coordinators. However, accepting interview invitations has become an applicant arms race (34, 35) . Over recent years, accepting residency invitations within minutes after they are received has become a top priority for applicantsperhaps even more so during the virtual season. Because of increased applications and fear of interview hoarding, available spots tend to fill within minutes of their release. This is particularly challenging for applicants in different time zones who receive invitations in the middle of the night. Additionally, the practice of some programs extending more invitations than available interview spots still occurs, as was experienced this year by some of the co-authors of this paper (35) (36) (37) .

Students now go to great lengths to optimize their ability to accept invitations (37) . For example, students have voluntarily removed themselves from valuable educational opportunities, such as procedures and surgical experiences, out of fear of failing to respond quickly enough (34) . Some coauthors now matched into radiology admit to pulling off of the road abruptly while driving in order to promptly accept invitations, putting their lives at risk. Finally, students who intentionally limit their time spent on devices to maintain healthier work-life balance and mental well-being often feel obligated to remain constantly tethered to their devices.

Prior to the virtual interview season, the topic of interview hoarding -accepting a greater number of interview spots than is statistically required to match -had received attention among GME leaders. The concept is well-demonstrated by data from the 2016 Diagnostic Radiology residency match, which reveals that 50% of all available interview spots were held by only 18% of well-qualified Diagnostic Radiology applicants (38) . As with over-application, the virtual interview season exaggerated this phenomenon. In December of 2020, the Chief Medical Education Officer for the AAMC warned of "a maldistribution of residency interview invitations" where "students in the highest tier receiv[ed] a larger number of interviews per person than in the past years" (39) . In other words, the most competitive radiology applicants held a disproportionate number of overall interview spots. The cause was likely multifactorial but included the perception of increased competition for positions, the lack of financial disincentive normally associated with additional interviewing, and the logistical ease of interviewing at more programs this cycle. As a consequence, many qualified applicants were not able to obtain an adequate number of interviews, as evidenced by an increase in applicants that participated in the Supplemental Offer and Acceptance Program (SOAP) this year (40) .

During traditional in-person interview cycles, as students received adequate numbers of invitations and began interviewing, they would reassess which to accept and how many they would ultimately need for a safe rank list. This enabled applicants to cancel unneeded interviews starting midway through the season. Additionally, students had the financial incentive to cancel because it saved hundreds or thousands of dollars in travel expenses. Because the cost of attending additional interviews during the virtual season was comparatively very low, fewer students cancelled interviews. A lack of cancellations combined with interview hoarding may have led to an unprecedented level of inequity in the 2020-2021 season. Without a system that limits or caps the number of interviews per applicant, recruitment inequity and participation in the SOAP process may only increase.

Due to virtual interviews, applicants lost the ability to critically appraise programs as one would typically do during in-person interviews. A virtual interview may offer relatively limited insights into a residency program and city, and a more limited perspective for the program about the medical student. Historically, the three most important factors affecting rank lists of students applying to radiology were the perceived happiness of current residents within the program, the geographic location of the program, and the academic reputation of the department (41) . Virtual interviews may have made it difficult to assess these factors, due to limited interaction with faculty and residents and inability to travel. Programs which adapted quickly, establishing an online presence and creating a virtual visiting experience, were viewed in higher regard by applicants as the ability to gauge these measures were minimized (20) . The effects of virtual interviews on overall fit and trainee satisfaction has yet to be seen, but will be something to examine in the future.

The virtual interview season of 2020-2021 demonstrated several key improvements in applicant equity including a reduced cost of interviewing, increased support for URiM applicants, and increased accessibility for applicants with disabilities or household responsibilities. At the same time, however, it may have also produced barriers to equity. Early recognition and discussion of these issues will provide opportunity for improvement. Residency stakeholders will have the opportunity to improve their virtual practices by intentionally working to enhance applicant equity (22, 23) . With reflection and a bit of work, virtual interviewing has potential to become the new normal in GME recruitment, improving access and opportunity for all residency applicants.

meded-April-14-Guidance-on-Medical-Students-Participationin-Direct-Patient-Contact-Activities

Final report and recommendations for Medical Education Institutions of LCME-accredited, U.S. osteopathic, and non-U.S. medical school applicants

Medical Student Debt -Is There a Limit?

Medical student education: debt, costs, and loan repayment face card for the class of 2020

Physician education debt and the cost to attend medical school: 2020 update

Cost of applying to residency questionnaire report

The socioeconomic diversity gap in medical education

Physician mental health: an evidence-based approach to change

Effect of medical student debt on mental health, academic performance and specialty choice: a systematic review

Underrepresented in medicine definition. AAMC. Diversity and inclusion web site

Current status of diversity by race, Hispanic ethnicity, and sex in diagnostic radiology

Physician workforce disparities and patient care: a narrative review

The plight of the minority resident physician-similar challenges in a different world

Can't we do better to support minorities in medicine

Minority resident physicians' views on the role of Race/Ethnicity in their training experiences in the workplace

Brave new world: challenges and opportunities in the COVID-19 virtual interview season

Mitigating bias in virtual interviews for applicants who are underrepresented in medicine

COVID-19 and the radiology match: a residency program's survival guide to the virtual interview season

Disabled medicine

The Etiology of Application Fever

Relationship between the number of residency applications and the yearly match rate: time to start thinking about an application limit?

Residency match during the COVID-19 pandemic: the clear and present danger of the remote interview

The urology match as a prisoner's dilemma: a game theory perspective

Are United States medical licensing exam step 1 and 2 scores valid measures for postgraduate medical residency selection decisions

A narrative review of the evidence supporting factors used by residency program directors to select applicants for interviews

Examining demographics, prior academic performance, and United States medical licensing examination scores

The impact of United States Medical Licensing Exam (USMLE) step 1 cutoff scores on recruitment of underrepresented minorities in medicine: a retrospective cross-sectional study

The numbers are in: our applications are only up »5% vs. last year. In: @jbcarmody, ed. The numbers are in: Our applications are only up »5% vs. last year. I had predicted a 20% increase due to systemic overapplication from COVID19. But I'd be happy to be wrong about that. Wonder what others are seeing

Q&A: Pandemic's lasting impact on the future of residency selection. AMA. Match Web site

Potential implications of COVID-19 for the 2020-2021 residency application cycle

The residency interview scheduling process: unintended consequences and a proposal for change

A Challenge to disrupt the disruptive process of residency interview invitations

APPLICANTS: think about what a residency program is telling you when they over-invite applicants to interview. In: @jbcarmody, ed. APPLICANTS: Think about what a residency program is telling you when they over-invite applicants to interview. If they are this selfish and inconsiderate now -and view you as so easily replaceable -do you really think it's gonna get better when you're actually in the program

First Come, First Served" serves no one: a call for change to residency interview invitation practices

I dream of Gini: quantifying inequality in otolaryngology residency interviews

Factors that affect national resident matching program rankings of medical students applying for radiology residency