key: cord-0787976-oi84pec3
authors: Bilgic, Elif; Okrainec, Allan; Valanci, Sofia; Di Palma, Adam; Fecso, Andras; Kaneva, Pepa; Masino, Caterina; Watanabe, Yusuke; Vassiliou, Melina C.; Feldman, Liane S.; Fried, Gerald M.
title: Development of a simulation curriculum to teach and assess advanced laparoscopic suturing skills using telesimulation: a feasibility study
date: 2022-01-08
journal: Surg Endosc
DOI: 10.1007/s00464-021-08880-6
sha: bfa9aac4571ededdf9dcbbbcceafef3e7dda5ece
doc_id: 787976
cord_uid: oi84pec3

BACKGROUND: Telesimulation helps overcome limitations in time and local expertise by eliminating the need for the learner and educator to be physically co-located, especially important during COVID-19. We investigated whether teaching advanced laparoscopic suturing (ALS) through telesimulation is feasible, effective, and leads to improved suturing in the operating room (OR). METHODS: In this prospective feasibility study, three previously developed 3D-printed ALS tasks were used: needle handling (NH), suturing under tension (UT), and continuous suturing (CS). General surgery residents (PGY4-5) underwent 1-month of telesimulation training, during which an expert educator at one site remotely trained residents at the other site over 2–3 teaching sessions. Trainees were assessed in the three tasks and in the OR at three time points: baseline(A1), control period(A2), and post-intervention(A3) and completed questionnaires regarding educational value and usability of telesimulation. Paired t-test was used to compare scores between the three assessment points. RESULTS: Six residents were included. Scores for UT improved significantly post-intervention A3(568 ± 60) when compared to baseline A1(416 ± 133) (p < 0.019). Similarly, scores for CS improved significantly post-intervention A3(756 ± 113) vs. baseline A1(539 ± 211) (p < 0.02). For intraoperative assessments, scores improved significantly post-intervention A3(21 ± 3) when compared to both A1(17 ± 4) (p < 0.018) and A2(18 ± 4) (p < 0.0008). All residents agreed that tasks were relevant to practice, helped improve technical competence, and adequately measured suturing skill. All residents found telesimulation easy to use, had strong educational value, and want the system to be incorporated into their training. CONCLUSION: The use of telesimulation for remotely training residents using ALS tasks was feasible and effective. Residents found value in training using the tasks and telesimulation system, and improved ALS skills in the OR. As the pandemic has caused a major structural shift in resident education, telesimulation can be an effective alternative to on-site simulation programs. Future research should focus on how telesimulation can be effectively incorporated into training programs.

Laparoscopic suturing is a skill required in a variety of minimally invasive surgical procedures (e.g., repair of hiatal hernias, bowel surgery), and when dealing with intraoperative complications. This skill, however, is challenging for surgical trainees to master, even with the utilization of simulation platforms such as Fundamentals of Laparoscopic Surgery (FLS™) for training in basic laparoscopic skills and certification [1, 2] .

FLS is a standardized educational program that includes didactic modules, and a manual and cognitive skills assessment component, designed to teach and evaluate the knowledge, skills and judgment that are fundamental to the practice of laparoscopic surgery [2] . FLS includes tasks related to suturing and knot tying with the usage of intracorporeal and extracorporeal techniques. There is ample evidence supporting the value of bench-top simulation to develop fundamental laparoscopic suturing skills, and FLS is a current requirement for certification by the American Board of Surgery [3] . Despite this, a survey of Program Directors (PDs) in the Fellowship Council reported that 60% of the PDs felt that graduating residents were not proficient in laparoscopic suturing [4] . This could be explained by the fact that skills required to perform laparoscopic suturing in the operating room (OR) may differ from the skills that are assessed by currently available programs such as FLS.

A previous needs assessment identified a gap between the skills that are important in the OR versus skills targeted by various simulation platforms and courses. One such skill is laparoscopic suturing [5] . Several laparoscopic suturing educational programs have been developed previously, using inanimate models, virtual reality simulations and animals [6, 7] . However, these models have several limitations in terms of cost, availability and validity evidence to support them as measures of advanced laparoscopic suturing (ALS) skills. In order to address this need, our team developed ALS tasks for free needle suturing, using widely available and low-cost materials [8, 9] . The developed tasks were based on various needs assessments, and were meant to be more reflective of the complexities of laparoscopic suturing in the OR [10] .

Simulation training has been shown to be an effective and efficient way for trainees to acquire surgical skills, and to use these skills in the clinical setting [11] . Simulation has been widely adopted due to concerns regarding lack of consistency in teaching methods, and concerns regarding patient safety and ethics [12] . Therefore, with simulation training, trainees have a platform to acquire skills outside of the clinical setting, so that when they operate on patients, they already have a skillset to build upon. For example, Antosh et al. found that training with the FLS tasks improves suturing skills in the OR for gynecology residents [3] .

Even though simulation training has many benefits, there are some difficulties when it comes to the actual implementation of a simulation program [13] . One of the main obstacles is related to the time constraints of the educators; it can be difficult to find educators for the sessions due to their clinical commitments, or lack of available educators able to teach the targeted skills. This is where telesimulation comes into play; it is a way for medical educators to reach trainees/ surgeons in other locations using telecommunication technologies [14] . This system removes the requirement for the learner and the educator to be physically in the same location and allows flexibility regarding when and how the training sessions could be scheduled. Hence, telesimulation has a lot of potential to be used for surgical skills training, without the need for the physical presence of an experienced and knowledgeable educator [15] . These benefits are especially important and amplified with the strict limitations to on-site simulations due to the Covid-19 pandemic [16] . Therefore, the purpose of our study was to investigate whether training using ALS tasks through telesimulation is feasible and has educational value at improving skills in the OR.

This was a multicentre prospective single group feasibility study (study design in Fig. 1 ). Senior general surgery residents (postgraduate year (PGY4,5) at McGill University and the University of Toronto were invited to participate.

The study was conducted at McGill University and University Health Network (UHN).

ALS simulation tasks previously developed at McGill University were 3D printed and used [8, 9] . These tasks are Needle Handling (NH), Suturing Under Tension (UT), and Continuous Suturing (CS). For NH, trainees are provided with a circle with 6 holes at different angles and are asked to go through each hole from outside to inside, requiring trainees to change the needle angle according to the location and angle of the hole. In UT, trainees perform 3 interrupted sutures on a tissue that is under tension (e.g., simulating laparoscopic suturing in foregut procedures). In CS, trainees are asked to perform continuous suturing that resembles closure of an enterotomy (e.g., simulating laparoscopic suturing in bowel procedures). The tasks were fitted to and performed in the widely available standard FLS trainer box.

Proficiency benchmarks were developed for these tasks previously using performance of laparoscopic surgeons experienced at laparoscopic suturing [8] . The simulated skills were assessed for efficiency (time) and errors. In addition to these metrics, in order to standardize the feedback that trainees received during the telesimulation sessions, we used Formative Feedback Tools (FFT) that were developed for each task separately in a prior study [17] . Global Operative Assessment of Laparoscopic Skills (GOALS) was used to assess skill during free suturing performance in the operating room. GOALS is a widely used assessment tool with validity evidence to evaluate general skills in laparoscopic surgery [18] .

At baseline, trainee performance in the OR was assessed in a procedure that requires laparoscopic suturing, such as a laparoscopic Nissen fundoplication, paraesophageal hernia repair, or gastric bypass, while suturing with a freeneedle. The attending surgeons of each case evaluated the performance of the trainee for the suturing portion of the procedure right after the completion of the case. The trainees were assessed when performing the simulated advanced laparoscopic suturing tasks using the time and error metrics (directly by a trained rater). The simulation and OR assessments were done within the same week. At the end of the simulation assessment, trainees were asked to fill out a survey (Survey 1) regarding demographics, case exposure, and future plans for fellowship/practice.

After the trainees were assessed at baseline, they were assessed a second time after 1 month in the OR and simulator. During this month, trainees continued standard clinical and educational training with no simulation training intervention.

Overview Once participants were assessed 1 month after baseline, as they continued to follow their surgical programs' educational and clinical activities, trainees were asked to practice the ALS skills in a simulation lab with a remote educator. In each session, trainees were guided to reach time and error proficiency benchmarks. Additionally, FFTs were used as a feedback guide for the educators, in order to standardize the feedback trainees received.

The training sessions were done remotely, using Skype free videoconferencing software (Skype Communications SARL, Luxembourg; https:// www. skype. com/ en/), by an expert educator who could teach laparoscopic suturing skills (expert educator at one site remotely trained the study participant (resident) at the other site). Educators were experienced Minimally Invasive Surgery (MIS) surgeons or MIS fellows. Before the start of the telesimulation sessions, the educators were trained on the tasks and the assessments to be used during the training (time, error, and FFTs).

In the first remote session, trainees were provided with instructions on how to complete the tasks and how they will be assessed before practicing each task. After the initial session, trainees were given a time span of 1 month for completion of the intervention (depending on the availabilities of the remote educator and the trainee, training sessions were once or twice a week). This way, trainees were given enough time to complete the intervention, and we limited the impact on their clinical work. Training started with needle handling, and included a baseline assessment (for educator to understand the level of the trainee), one-on-one training (with assessments throughout), and a final assessment (trainees could also ask additional questions, or educators could provide additional feedback/tips). After needle handling, training continued with suturing under tension and continuous suturing. Finally, the three tasks were practiced a final time with the educator.

At the end of the telesimulation training, trainees were asked to fill out a survey (Survey 2) regarding system usability, educational value, audiovisual reliability and session quality, and adaptation and integration.

Trainees were assessed a third time in the simulator and OR using the same assessment tools as baseline and 2nd assessment, in a procedure using free-needle, through direct observation by the attending surgeon.

Research ethics board approval was obtained at McGill University and UHN and study participants provided written consent.

Descriptive statistics were used to describe the sample. Parametric (i.e., independent t-test) testing was used to compare the final scores and the change between the two groups. Paired t-test was used to compare the differences within the groups. Results were analyzed using SPSS and Excel software. Data is presented as Mean ± Standard Deviation (SD).

Seven residents were recruited. Due to COVID-19 pandemic, 2 were unable to complete the study One only missed assessment A3; therefore, their data was included in the analysis (6 residents median age 30; 83% male). Three residents were from McGill University (1 PGY4, 2 PGY5), 3 from University of Toronto (3 PGY4), and 5 were FLS certified. In terms of future plans, 2 said they would like to pursue a MIS fellowship, 1 MIS/bariatrics, 1 colorectal, and 1 trauma and bariatrics (data for 1 was not available). Residents had varying experience with performing laparoscopic suturing using free-needle in the OR, ranging from 1 to 5 cases per year (3 residents), 6-10 (1 resident), 11-25 (1 resident), and 26-50 (1 resident). The same trend was noted for prior experience with performing laparoscopic suturing using free-needle in simulation.

Scores for suturing under tension improved significantly post-interventionA3 (568 ± 60) when compared to baseline A1(416 ± 133) (p < 0.019) (Table 1) . Similarly, scores for continuous suturing improved significantly post-intervention A3 (756 ± 113) vs baseline A1 (539 ± 211) (p < 0.02). For intraoperative assessments, scores improved significantly post-intervention A3 (21 ± 3) when compared to both baseline A1 (17 ± 4) (p < 0.018) and control A2 (18 ± 4) (p < 0.0008). Needle handling scores improved during the control period, but there was no significant further improvement post-intervention. All residents reached pre-defined time-error proficiency benchmarks during training and overall Formative Feedback Tools scores were observed to improve.

Residents were surveyed regarding the educational value of the tasks (Survey 1). Residents 'agreed' or 'strongly agreed' that all three tasks were relevant to practice, helped improve technical competence, and adequately measured advanced laparoscopic suturing skills.

We explored trainees' perceptions about telesimulation sessions (5 residents completed Survey 2). Except for 1 resident, none of them had prior experience using telemedicine equipment or training with live intra-operative surgical videos. In terms of usability of the telesimulation system, 4 residents agreed or strongly agreed that they would use telesimulation frequently and that the system was easy to use. Additionally, all 5 residents agreed or strongly agreed that various functions of the telesimulation system were well integrated, most people would learn to use this system very quickly, and felt very confident using the system.

Video and audio quality of the telesimulation system was rated on a scale from 1 to 5 (1 poor, 3 adequate, 5 reference quality); overall, the scores for each item ranged from 3 to 5, with video delay, audio echo, audio/video synchronization, stability, and overall appropriateness for telecoaching being rated 4 or 5 (reference quality). The educational value of the telesimulation sessions was rated by 4 of the 5 residents as "agreed/strongly agreed" that it fulfills trainees' educational needs, improves communication between trainee and attending surgeon, and facilitates learning and intra-operative decision-making. All residents agreed or strongly agreed that telesimulation is an effective adjunct to current surgical training, facilitates teaching intra-operative technical skills, facilitates the acquisition of surgical knowledge, allows the trainee to receive additional and more meaningful feedback on their performance or others' performance, and improves overall resident performance.

Finally, for trainee perceptions regarding the adaptation and integration of telesimulation, all residents agreed that they would want telesimulation to be incorporated into their training program and hospital infrastructure. Additionally, 4 of the 5 residents reported that if telesimulation was available, they would use it all the time. When residents were asked about barriers for implementation of telesimulation, 4 agreed or strongly agreed that time constrains and resources (personnel, maintenance, equipment), and 2 agreed that cost would limit the implementation.

In order to help overcome institutional differences in the simulation set-up, we used a standardized set-up procedure previously used for FLS tele-proctoring studies [19, 20] . Additionally, a member of the research team was present at both sites during the sessions and troubleshooted any technical issues. We had some technical issues at the beginning (related to connection, and computer malfunctioning at the start of the session), which were resolved, and we did not encounter technical issues after 1-2 usages.

Scheduling of the telesimulation sessions was done by the lead author, supported by P.K. and C.M. Overall, the technology worked effectively after a short learning phase. However, scheduling required a lot of communication and flexibility from educators and trainees, and delays or rescheduling did happen due to prolonged OR times or other emergencies involving the educator or the trainee.

Trainees were motivated to take part in the study, as it gave them an opportunity to practice advanced laparoscopic suturing skills one-on-one with an educator, which is not an opportunity they get often. Also, since the trainees were senior residents and some were looking for MIS fellowships, this allowed them to network with the educator from the other site. Also, educators were interested in teaching the residents other laparoscopic suturing techniques that they perform at their site, which the residents responded to with enthusiasm.

Based on the results of our study, telesimulation is a feasible option to train the residents in advanced laparoscopic suturing, and is an effective learning tool, leading to significant improvement in the trainees' performances in both the OR and simulation settings. All residents found telesimulation easy to use, that the provided video and audio were of high quality, that the curriculum had strong educational value, and residents want the system to be incorporated into their training curriculum. In addition to these benefits, authors observed that telesimulation sessions created an opportunity for residents to practice advanced laparoscopic suturing with an educator from another site, allowing for crossinstitutional networking and mentorship.

Telesimulation has been shown in prior studies to allow for training and assessment of trainees (or surgeons learning a new skill) from remote or resource-limited locations, providing flexibility for trainees and educators, overcoming travel costs, and allowing for networking and collaboration across institutions, which was evident from our study as well [21] . For example, a study that used telesimulation to train 16 surgeons from Botswana, Africa on the FLS tasks found that surgeons significantly improved their laparoscopic skills in the tasks compared to surgeons in the self-practice group [15] . Even though there were no assessment of operative performance, this study has shown the educational value of telesimulation, when one-on-one simulations with the physical presence of an educator cannot be possible.

The value of telesimulation, as specified above, has been amplified with the Covid-19 pandemic. Residency programs have been affected in multiple ways, including limitations in trainees' clinical and case exposures and on-site educational sessions, including in-person simulation training [16, 22, 23] . As a result, many surgical educators and programs have tried to find ways to overcome these limitations caused by the pandemic. Currently, many institutions have delivered grand rounds, workshops and other didactic teachings virtually [24] . However, when it comes to skills acquisition and assessment, including for laparoscopy, the training of surgical residents was hindered. Hence, programs have been investing in ways to provide simulation training outside of the hospitals or simulation centers [25] . One of these options is telesimulation, which can have a beneficial role within a program to conduct simulation training sessions from a distance. Telesimulation might usually be applied to training of residents/surgeons in remote or resource-limited locations, where expertise in a particular skill or procedure is lacking. However, telesimulation can also be used within the same program, where the trainees can be provided with the necessary simulation materials and have telesimulation sessions (in groups or individually) with their senior colleagues or faculties [26] . This can help address logistical and scheduling issues and becomes especially important with the social distancing rules and limits in the number of individuals allowed in a simulation laboratory at one time.

In terms of the telesimulation system setup, we used the standardized set-up that our study author A.O. used for FLS tele-proctoring [19] The system included a laptop with necessary applications already loaded (Skype), and splitter video cables/devices to be used to connect with the same standard FLS box trainer at both sites, to conduct the sessions. This allowed us to limit the potential effect of setup differences on the trainee performance and allowed us to be consistent across all of the trainees. The equipment was set up before the 1st session, and was not modified until the end of the study.

The simulation performance metrics, based primarily on time and error, are commonly used in simulation studies, and have been shown to be feasible, reliable and well supported by validity evidence. Specifically, evidence for validity has been published for the advanced laparoscopic suturing tasks used in this study [27, 28] . Formative Feedback Tools are assessment instruments that were previously developed for our simulation tasks, and validity evidence has been established for them as measures of laparoscopic skills in these tasks [17] . In addition, a pilot study that was presented at a Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) conference showed the value of FFTs in providing feedback to improve trainee skill [29] . We have published a comprehensive literature review of available assessment tools for laparoscopic suturing skills in the OR, and procedures that commonly require laparoscopic suturing [30] . Based on the results, GOALS had the most validity evidence in the context of operative assessment through direct observation by an attending surgeon, which is why we have used GOALS for intraoperative assessment in this study.

In terms of limitations, our study had 6 participants, and due to the COVID-19 pandemic, recruitment of more participants was not possible. Nonetheless, this was a feasibility study, and we were interested in exploring how telesimulation training could be structured and implemented, and the perceptions of the trainees. Additionally, it could be important to explore the logistics of how telesimulation might be used within or across institutions, or for trainees to practice from home (e.g., if they could be provided with the necessary resources). Also, our study was able to show skill improvement in the simulated setting and transferability of these skills to the operating room. Nonetheless, due to our limited sample size and single group design, we need further studies to confirm this transfer of skill. Hence, future studies could examine widespread adoption of telesimulation within a larger curriculum and explore the impact of telesimulation on skills acquisition in the OR.

Videoconferencing technologies have been proposed in telementoring, telemedicine, and tele-proctoring learning activities because they are a relatively inexpensive solution and can enable virtual education anywhere. Telesimulation is a feasible and effective means of providing technical skills training. This study's implementation of a curriculum to teach advanced laparoscopic skills using telesimulation resulted in improvements in performance, not just in a simulator, but transferrable to the operating room. Although the COVID-19 pandemic imposed challenges for accrual of trainees in this study, this also created some urgency in developing novel ways to train residents and rapidly accelerated the adoption of videoconferencing platforms across teaching and learning. In the future, researchers should investigate other skills that could effectively be taught by telesimulation, and how to most effectively incorporate telesimulation into training programs, addressing logistical and scheduling challenges.

Funding There was no funding or financial support for this work.

Disclosures Dr. Fried is a consultant to Theator. Dr. Okrainec reports honoraria fees for speaking and teaching from Medtronic, Ethicon, and Merck. Pepa Kaneva, Caterina Masino, and Drs Elif Bilgic, Allan Okrainec, Sofia Valanci, Adam DiPalma, Andras Fecso, Yusuke Watanabe, Melina C Vassiliou, Liane S Feldman, and Gerald M Fried report no relevant conflicts of interest or financial ties to disclose related to the content of this research.

Laparoscopic suturing as a barrier to broader adoption of laparoscopic surgery

Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery

Blinded assessment of operative performance after fundamentals of laparoscopic surgery in gynecology training

General surgery residency inadequately prepares trainees for fellowship: results of a survey of fellowship program directors

Identifying the need for and content of an advanced laparoscopic skills curriculum: results of a national survey

The use of simulation in the acquisition of laparoscopic suturing skills

A simple, lowcost platform for basic laparoscopic skills training

Simulation Committee of the Association for Surgical E (2016) Multicenter proficiency benchmarks for advanced laparoscopic suturing tasks

New models for advanced laparoscopic suturing: taking it to the next level

What are the training gaps for acquiring laparoscopic suturing skills?

Fundamentals of laparoscopic surgery simulator training to proficiency improves laparoscopic performance in the operating room-a randomized controlled trial

Competency-based postgraduate training: can we bridge the gap between theory and clinical practice

Telesimulation: an innovative tool for health professions education

Telesimulation: an effective method for teaching the fundamentals of laparoscopic surgery in resource-restricted countries

Surgical simulation in Africa: the feasibility and impact of a 3-day fundamentals of laparoscopic surgery course

Surgical education in the time of COVID: understanding the early response of surgical training programs to the novel coronavirus pandemic

Establishing meaningful benchmarks: the development of a formative feedback tool for advanced laparoscopic suturing

A global assessment tool for evaluation of intraoperative laparoscopic skills

Remote FLS testing in the real world: ready for "prime time

Feasibility of remote administration of the Fundamentals of Laparoscopic Surgery (FLS) skills test

Prospective randomized crossover study of telesimulation versus standard simulation for teaching medical students the management of critically ill patients

Impact of the COVID-19 pandemic on surgical training and learner well-being: report of a survey of general surgery and other surgical specialty educators

Surgical training in the midst of a pandemic: a distributed general surgery residency program's response to COVID-19

Restructuring of a general surgery residency program in an epicenter of the coronavirus disease 2019 pandemic

Off-site training of laparoscopic skills, a scoping review using a thematic analysis

The sim must go on: adapting resident education to the COVID-19 pandemic using telesimulation

Goaldirected laparoscopic training leads to better laparoscopic skill acquisition

Evaluation of laparoscopic skills: a 2-year follow-up during residency training

Use of a formative feedback tool in place of an expert coach in laparoscopic suturing training: a randomised non-inferiority trial (Abstract)

A scoping review of assessment tools for laparoscopic suturing

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations