key: cord-1045605-zoldcz6w
authors: Ream, Margie A.; Stone, Robert
title: Virtual residency interview experience – the child neurology residency program perspective
date: 2021-09-30
journal: Pediatr Neurol
DOI: 10.1016/j.pediatrneurol.2021.09.016
sha: 336846ecf2bf4d4109ddcf8129f24f0f3053cbe0
doc_id: 1045605
cord_uid: zoldcz6w

BACKGROUND: The COVID-19 pandemic presented many challenges for graduate medical education, including the need to quickly implement virtual residency interviews. We investigated how different programs approached these challenges to determine best practices. METHODS: Surveys to solicit perspectives of program directors, program coordinators and chief residents regarding virtual interviews were designed through an iterative process by two child neurology residency program directors. Surveys were distributed by email in May 2021. Results were summarized using descriptive statistics. RESULTS: Responses were received from 35 program directors and 34 program coordinators from 76 programs contacted. Compared to the 2019-2020 recruitment season, in 2020-2021, 14 of 35 programs received >10% more applications and most programs interviewed ≥12 applicants per position. Interview days were typically 5-6 hours long and were often coordinated with Pediatrics interviews. Most programs (13/15) utilized virtual social events with residents, but these often did not allow residents to provide quality feedback about applicants. Program directors could adequately assess most applicant qualities but felt that virtual interviews limited their ability to assess applicants’ interpersonal communication skills and to showcase special features of their programs. Most respondents felt that a combination of virtual and in-person interviewing should be utilized in the future. CONCLUSIONS: Residency program directors perceived some negative impacts of virtual interviewing on their recruitment efforts but in general felt that virtual interviews adequately replaced in person interviews for assessing applicants. Most programs felt that virtual interviewing should be utilized in the future.

The COVID-19 global pandemic presented a vast array of new challenges for graduate medical education. Among these was the need to quickly implement solely virtual residency interview experiences for the 2020-2021 recruitment season. With the help of graduate medical education (GME) planning within institutions and swift efforts conducted within professional organizations, the interview season proceeded with only a short delay in applications opening to programs for review. However, the impact of virtual recruitment and opportunities it represents are still being defined.

The child neurology recruitment process presents unique considerations for programs and applicants. Interviews must be coordinated between two and sometimes three departments -pediatrics, adult neurology, and child neurology. Applicants must consider their education during different phases of training. Additionally, quality of life factors related to a program's location and culture are increasingly important [1] .

Prior to COVID-19 there were some reports comparing in-person and virtual residency and fellowship interview experiences. Most of these studies were limited by occurring at a single institution and in a single -and often times procedural -specialty. But available data suggests that there is "universal gratitude" among applicants for having the option for virtual interviews [2] . At the same time, 15% of applicants felt that a virtual format negatively impacted their self-presentation and 34% said that it negatively impacted their ranking of the program during a season in which they interviewed in-person at J o u r n a l P r e -p r o o f other programs [2] . Interview format did not affect program ranking of applicants or match rate for anesthesiology and ophthalmology [3, 4] , despite the perception at least within a cardiothoracic surgery fellowship, that it did [5] .

Students and residents in a variety of specialties prefer in-person interviews but still feel that virtual should be an option even if they themselves wouldn't necessarily opt for it [6] . Cost, travel distance, and time away from education are factors favoring virtual interviews, while the desire to interact directly with residents and faculty and to assess potential quality of life in a location are factors in favor of in-person interviews [6] .

Removing the barriers of in-person interviewing allows applicants to interview with more programs [3, 7] . But of applicants who completed a virtual interview, 17% still incurred the time and expense to travel to the city to visit informally prior to matching [4] .

There are advantages and disadvantages for programs too. Some programs felt that they were limited by the virtual format in showcasing their attributes [5, 8] . For example, it was difficult to adequately convey facilities through a virtual tour [5] . Program directors often found it difficult to assess an applicant's "fit" for their program through virtual interviews [7, 8] . However, virtual interviews may require less faculty time than in-person interviews [2] . The majority of applicants and program directors have a favorable view of continuing virtual interviews, at least in some capacity, when pandemic-related restrictions are lifted [5, 8, 9] .

We sought to better define the experience of child neurology residency programs with virtual interviews during the 2020-2021 residency recruitment season.

In May 2021, child neurology residency program directors (PDs), program coordinators (PCs), and chief residents (CRs) in the United States were recruited to participate. A list of programs and contact information was obtained from the Child Neurology Society website listing of Child Neurology Training Programs, and 76 child neurology program director-coordinator dyads were contacted through direct email. CRs were not contacted directly, but the invitation to PD/PCs contained a link for a separate resident survey to be distributed to any CRs who were involved in 2020-21 recruitment efforts. 

We received responses from 35 PDs, 34 PCs and 15 CRs out of the 76 programs we attempted to contact, resulting in a response rate of 46% for program directors and 45% for program coordinators. Response rate for chief residents could not be calculated as we could not control for the number of residents who received the survey invitation from their directors and/or coordinators. Additionally, PCs and PDs were not instructed to send the chief resident to a specific number of residents. CR respondents were more likely to be from larger programs than the other respondents (weighted average of 3.4 positions per program for CRs compared to 2.3 for PDs and PCs).

The numbers of positions offered in the 2020-2021 recruitment season compared to 2019-2020 were similar for our respondents, which is consistent with data reported by reported that 26% ranked ≥10% more applicants than the previous year.

Program coordinators most often reported fewer cancelled interview appointments, and those that were cancelled were easier to fill than previous years. The interview days were coordinated with pediatrics interviews in 23 of 33 programs (either same day [18/33] or consecutive days [5/33] ). The interview day contained a variety of activities with the most common being an opportunity to meet residents (30/34) and a program overview presentation (28/34). Table 1 specifies what types of live (synchronous) content was offered as described by PCs. Data extracted from narrative comments provided by 21 PCs indicated that the majority of interview days were single day experiences (15/21) that ranged from < 5 hours (4 programs) to 8-9 hours (3 programs), J o u r n a l P r e -p r o o f with a mode of 5-6 hours in a day (8 programs) . On the other hand, 6/21 programs included multi-day experiences that ranged from 8 to 13 hours combined.

Social events required significant coordination for the 2020-2021 interview season with reliance on platforms that were new for many programs. All but one chief resident reported resident participation in virtual social events and most (13/15) felt that the quality of social interaction in a virtual platform was inferior to that of an in-person experience. CR comments indicated that it was difficult to have good conversations and get to know the applicants as well as showcase the city. However, smaller groups and breakout rooms seemed to be the most useful approach. Program participation in social media grew significantly between the 2019-2020 and 2020-2021 interview seasons (increased from 5 to 13 of the 15 programs with CR survey responses), and the majority of CRs (9/15) indicated that they had a major role in maintaining social media accounts. Despite heavy resident involvement, only 2 out of 15 residents reported that they were able to provide quality feedback to their programs on applicants.

One commented that they could provide feedback only on applicants who were clear outliers from the group, either positive or negative, and another commented that resident ability to get to know applicants improved over the season as they became more familiar with the process. 

CRs and PCs were more likely to predict that in-person interviewing in the future will require more of their time than virtual recruitment. PDs were more likely to predict that However, through free text comments, all groups raised concerns about equity, and judgement that would favor applicants who could visit the program in-person. Given this concern, respondents indicated the need for a uniform approach for all applicants.

Several respondents suggested a virtual interview day followed by an in-person second J o u r n a l P r e -p r o o f visit that could be voluntary, conducted after rank lists are completed or completed in a manner that blinded program leadership from the identity of visiting applicants.

This study investigated the residency program perspective on virtual recruitment during the 2020-2021 recruitment cycle by surveying Child Neurology PDs, PCs, and CRs.

With the change to virtual interviewing, there were many uncertainties faced by programs including: (1) whether applicants would apply to more programs (thus necessitating interviewing more applicants), (2) Among child neurology programs that responded to our survey, 40% saw a >10% increase in applications received. Because less than 3% of programs reported a 10% decrease in applications received, the findings suggest an overall increase in applications per program. The lack of extrinsic limitations on accepting interview offers, such as money and travel time, that is afforded by virtual interviews, likely contributed to the increase in applications [12] . Likewise, programs increased the number of applicants they interviewed, with 82% interviewing ≥12 applicants per position.

While child neurology remains a small field, minor changes in application, interview, and ranking trends could have significant effects on programs. NRMP data indicate that the total applicants ranking child neurology positions in the 2021 and 2020 Matches were J o u r n a l P r e -p r o o f 218 and 213, respectively [10] . While the number of applicants increased by only 5, the number of positions ranked by applicants increased from 1760 to 2109 between 2020 and 2021, which corresponds to an increase from 8.3 to 9.7 programs ranked per applicant, indicating that applicants ranked more positions [10, 11] . As a result, programs needed to rank an additional 1.9 applicants per position, or 7.9 applicants per position compared to 5.9 in 2020, in order to fill in 2021 [10, 11] . The "application inflation" has been seen in other specialties including obstetrics, maternal fetal medicine and dermatology [7, 8, 12] and can disadvantage less competitive applicants, ultimately leading to more unfilled positions nationally.

Overall, 92.5% of categorical child neurology positions were filled in the main 2021

Match [10] . This fill rate is lower than the 2021 fill rates for pediatrics and adult neurology positions which were 98.6% and 98.2%, respectively. The number of unfilled categorical child neurology residency positions increased from 8 in 2020 to 12 in 2021.

But given that the 2021 child neurology fill rate was within the range of the last 5 years (91.7-96.2%), we cannot make conclusions about the presence or impact of application inflation on child neurology at this point in time [10, 11] .

Increasing numbers of interviews is particularly challenging for child neurology residency because of the need to coordinate with pediatrics and, for some programs, adult neurology programs. Through our survey, we found a wide range in the length of interview days, and in how programs combined with Pediatrics. The majority of programs had a combined, single interview day with Pediatrics that was typically around sitting in front of their computer. Most programs included a synchronous (live) orientation and meetings with the residents, but less than half of programs included a live tour (or prerecorded tour with live comments).

Social interactions and the ability to convey quality of life and cultural aspects of a program and surrounding city are particularly challenging during a virtual interview experience, despite these factors being increasingly important to child neurology applicants [1] . In our survey, the majority of residents felt that the virtual social events were not as high quality as past in-person events. Some commented that the sessions were more of an informal question and answer session than an opportunity to learn about one another. As such, the vast majority of CRs did not feel that they could provide quality feedback to programs about the applicants (13/15). The use of small breakout rooms seemed to be helpful, but overall, the quality of applicant-resident interactions was likely a major drawback to virtual interviewing for both programs and applicants.

As programs enter another season of virtual recruitment and consider the future use of virtual recruitment, it is important to consider the amount of time residents spend contributing to social media and other recruitment efforts, particularly if these will result Program directors generally felt that virtual interviews were comparable to in-person interactions for assessing most applicant qualities, with the exception of interpersonal communication skills, which were better assessed in-person. It is noteworthy that interpersonal communication skills were also the most important characteristic to be evaluated during an interview. The majority of PDs (25/34) also felt that it was harder for them to highlight unique aspects of their programs, which indicates that virtual interviewing seemed overall less effective than in-person. Thus, it is reasonable to infer that overall, the interview process was negatively impacted by the virtual format.

Nationally, residency programs are struggling to determine the best recruitment strategy after pandemic restrictions are fully lifted. Our survey respondents seemed to favor a combination of both virtual interviews and in-person visits during the recruitment season. Through free text comments, some respondents acknowledge that allowing applicants to choose either virtual or in-person interactions could lead to inequity because applicants from further away or with fewer resources are less likely to travel.

As a result, a standardized experience that incorporates virtual interviews and removes any possible influence of in-person visits from a program's rank list may be able to maximize equity and flexibility for applicants while still allowing programs an opportunity to demonstrate their culture, facilities and other unique aspects to interested applicants.

Applicants could use the virtual interview to prioritize programs and allow them to concentrate their resources on visiting programs in which they have a genuine interest.

J o u r n a l P r e -p r o o f As long as there are minimal constraints on the number of interviews an applicant accepts, programs will likely face "application inflation." Professional societies and medical schools could collaborate on how to advise medical students when planning their applications. Standards within fields could also be agreed upon by program director groups, as has been done with obstetrics where there is an agreed upon final application deadline, a standard 72 hours in which applicants have time to reply to interview offers and a deadline for programs to notify applicants of their status (interview/no interview/waitlist). A ticketing system through which applicants have a limit on the number of interviews they can accept, with an increased allowance for couples matching, has been proposed but not implemented [13] . Such an interview cap could help candidates be more selective in their application process, improve the chances of less competitive applicants matching and reduce some of the burden on programs. However, limiting applications also limits program exposure to applicants who might discover a great fit with a program they might not have considered had they been more selective with interviewing. Consideration of an interview cap would require further discussion among stake holders in the national child neurology residency training community.

There were several limitations to this study. In an attempt to maximize completion of the surveys, we chose just 10 questions for each group, so the amount of information we could obtain was limited. We did not have the updated contact information for all programs, so a small number of programs may not have been able to participate in the study. Our response rates were high for PD and PCs, but was low for CRs as only 15

CRs responded to the survey. Therefore, our results may not be indicative of their J o u r n a l P r e -p r o o f broader perspective. Given the relatively small number of respondents from each group, additional analyses were not performed.

If virtual residency interviewing continues, programs may need to consider increasing the number of applicants interviewed as applicants may continue to apply to more programs. Most programs combined Child Neurology and Pediatrics interviews on one day, and the interview experience was most commonly about 6 hours long, including a live orientation and meeting with residents. In the future, programs may need to rethink how to structure interactions with residents to make them more useful and personal.

Overall, our surveys indicated that virtual interviews may limit some aspects of programs' and applicants' assessments of each other, although there are also advantages to virtual interviews. As programs gain experience with virtual interviewing, many may want to consider how to incorporate virtual experiences in the interview process in an effort to reduce the time and expense associated with in person interviews incurred by both applicants and programs.

J o u r n a l P r e -p r o o f 

Child Neurology Applicants Place Increasing Emphasis on Quality of Life Factors

Videoconference Interviews for an Adult Reconstruction Fellowship: Lessons Learned

Residency interview video conferencing

Comparison of web-based and face-to-face interviews for application to an anesthesiology training program: a pilot study

A Comparison Between In-Person and Virtual Fellowship Interviews During the COVID-19 Pandemic

Perception of medical students and residents about virtual interviews for residency applications in the United States

Effect of virtual interviewing on applicant approach to and perspective of the Maternal-Fetal Medicine Subspecialty Fellowship Match

Maternal-fetal medicine program director experience of exclusive virtual interviewing during the coronavirus disease 2019 pandemic

Program and candidate experience with virtual interviews for the 2020 Complex General Surgical Oncology interview season during the COVID pandemic

National Resident Matching Program, Results and Data: 2021 Main Residency Match®

National Resident Matching Program, Results and Data: 2020 Main Residency Match®

Improving the residency application process with application and interview caps

A Novel Ticket System for Capping Residency Interview Numbers: Reimagining Interviews in the COVID-19 Era