key: cord-0696980-nyr16yja
authors: Sood, Akshay; Pollard, Charles; Kalishman, Summers; Assad, Nour; LeSuer, Kyla; Khattar, Ramni; Soller, Brian; Myers, Orrin
title: Telementoring of Healthcare Teams in the Care of Miners
date: 2020-12-22
journal: ATS scholar
DOI: 10.34197/ats-scholar.2020-0073oc
sha: 6366087fe4fb121c4a2a281c01fe38c4e474ad9e
doc_id: 696980
cord_uid: nyr16yja

Background: Given the reemergence of pneumoconiosis in the United States, there is a tremendous need to train rural professionals in its multidisciplinary management. The Miners’ Wellness TeleECHO (Telementoring Extension for Community Health Outcomes) Program in New Mexico, United States, provides longitudinal multidisciplinary telementoring to professionals taking care of miners. The impact of this approach has not been previously evaluated. Objective: To examine the change in self-efficacy of professionals taking care of miners and participating in the TeleECHO Program. Methods: This is a 12-month longitudinal study involving clinical and nonclinical professionals caring for miners. The study outcome was the change in self-efficacy scores, using a customized instrument of 14 measures grouped into three domains: clinical, medicolegal, and soft skills. The primary outcome used a retrospective pre–post design that collects “pretest” data at the postintervention timeframe. Results: Participants reported significant improvements in 10 of 14 items (P < 0.05) and a significant decline in 1 of 14 items (with respect to their ability to interpret pulmonary function test results, P < 0.001) since their start dates in the program. Subjects also reported significant improvement with respect to their scores for all three domains and for the 14-item total score (P ⩽ 0.01). Existing participants and clinical professional groups demonstrated greater improvement in selected items than fresh participants and nonclinical professional groups, respectively. Conclusion: This study is the first in a stepwise approach to determine the benefit of participating in a multidisciplinary telementoring intervention by improving participant self-efficacy in caring for miners with complex mining-related diseases. Our study finding represents a potential solution to a growing access-to-care gap for miners with pneumoconiosis.

change in self-efficacy of participants, using a virtual community-of-practice approach in telementoring rural professionals taking care of miners in the United States. The outcome corresponds to level 3 of the Moore expanded framework for continuing medical education (CME) activities (8) . If effective, it provides an exciting opportunity for greater investment in interprofessional telementoring to foster collaborative healthcare practices in rural and remote mining communities as an approach toward countering the resurgence of pneumoconiosis.

This is a 12-month longitudinal study of participants involved in the TeleECHO Program, a novel community-university partnership between the University of New Mexico School of Medicine, Albuquerque, New Mexico, and its rural partner, the Miners' Colfax Medical Center, Raton, New Mexico. Together, these New Mexico sites constitute the hub site of experts. The spoke partner sites, located across the pneumoconiosismortality hot-spot regions of the United States, include clinicians, respiratory therapists, home health professionals, benefits counselors, lawyers or attorneys, and others, including policy-makers, administrators, and mine-safety officers (4) . The hub and the spoke partners together form a virtual community of practice.

As published previously (4) , the TeleECHO sessions are scheduled at the same time twice every month for 75 minutes, adhering to a standard format. After an initial 10 minutes of introduction and announcements, an invited expert delivers a 15-minute didactic presentation followed by a 20-minute, facilitated question-answer session and a 30-minute, interactive, facilitated case discussion. This format, employing adult-learning principles, focuses more on active learning through discussion than on didactic training. Participants can avail CME credits without charge upon completing the CME survey. A multidisciplinary curriculum committee frequently reviews the feedback provided to formulate a structured curriculum.

Irrespective of whether they present the case or not, participants can view the didactic and case discussions, provide information and insight from their own experience, and receive mentoring from the expert panel and their peers. Outside the program sessions, participants have access to experts or peers at the hub or other spoke sites, by telephone or e-mail, for urgent consultation requests. Over time with iterative practice and feedback, participants gain additional expertise and become more independent. Recorded and archived sessions are available from the program's website without charge.

Since July 2016, the program has used the ECHO model to provide structured long-term telementoring in the care of miners. This approach contrasts with telemedicine, in which experts provide short-term care to individual patients. This approach also differs from traditional lectures or webinars because it provides a real-time, interactive discussion of cases with expert panels. By contextualizing discussions, the program fulfills key learning theories, such as deliberate practice (9), social cognitive theory (10) , and situated learning and communities of practice (11) . As published previously (4), the five key principles of the ECHO model include the following: 1) use of Internet technology to leverage resources; 2) use of a diseasemanagement model shown to be effective in other diseases by sharing best practices (12) (13) (14) (15) ; 3) use of the principle of case-based learning; 4) creation of a community of practice, which emphasizes reciprocity in knowledge transfer, acknowledging that all participants bring some unique expertise; and 5) use of an Internet-based database to monitor outcomes. For the curriculum outlining individual didactics during the study timeframe, see Table E3 in the data supplement.

The study outcome was the change in self-efficacy scores, using a customized instrument created by a multidisciplinary curriculum and evaluations committee, based on the review of the literature and feedback from individual stakeholder groups. Self-efficacy, or a provider's perceived ability to recognize and manage essential components of care of complex miners' diseases and to serve as a local expert in these diseases for other community professionals, was assessed, with assessment adapted from previously validated instruments (16) . Participants rated each of the 14 self-efficacy measures on a scale of 1-7, in which "1" indicates "none" or no skill and "7" indicates "expert" or ability to teach others. The 14 measures were grouped into three domains: a six-item clinical-skills domain, a three-item medicolegal-skills domain, and a five-item soft-skills domain. The self-efficacy instrument, provided in Table  E4 , was not fully validated in the current study. The change in self-efficacy items, separately calculated since the subject's start date in the program (using the retrospective pre-posttest method with both pretest and posttest data obtained at the 12-month study time point) and over a 12-month study timeframe (using the traditional pre-posttest data method with pretest and posttest data obtained at the 0-and 12-month study time points, respectively) was calculated. For our primary study outcome, we chose a priori the retrospective pre-post design that collects "pretest" data during the postintervention timeframe at 12 months for the following two reasons. 1) By encouraging participants to reflect back and rate self-efficacy before participation in the ECHO Program with the benefit of hindsight, the retrospective pretest evaluation helps reduce the bias of selfevaluation (13) . This approach is useful, as ECHO evaluators have found that participants often do not realize how much they did not know until after they have participated in the program (17) . 2) Several prior ECHO outcome evaluations have demonstrated that, as compared with traditional pretest scores, retrospective pretest scores have lower mean and/or standard-deviation values, yielding greater power for analysis for small-sized comparison studies (18, 19) .

Data were collected during the 1-year timeframe of September 12, 2018, to September 18, 2019, using Research Electronic Data Capture (Vanderbilt University), a secure Web application for building and managing online surveys and databases.

Data collected were analyzed using Statistical Analysis Software version 9.4 (SAS Institute, Inc.). Wilcoxon signed rank sum tests for paired measurements were used to analyze change in continuous outcomes. Subgroup analyses for change scores were analyzed using t tests. Comparison groups included clinical professional groups versus nonclinical professional groups and existing versus fresh participants. Clinical professional groups include clinicians, respiratory therapists, and home health professionals (mostly nurses). Nonclinical professional groups include lawyers or attorneys, benefits counselors, and others. Fresh participants were defined as those who first attended the community of practice in or after the summer of 2018 (defined as May 9, 2018, and onward) versus existing participants (defined as those who had first attended any time between July 1, 2016, and May 8, 2018). Existing participants thus had greater cumulative participation and experience with the TeleECHO Program than fresh participants (11.4 ± 9.8 vs. 4.6 ± 4.6 total sessions before and/or during the study timeframe; P = 0.03). The cut-point date was chosen on the basis of the date of funding by the sponsor, which allowed the frequency of the TeleECHO Program to be raised from monthly to twice a month. A two-tailed P value < 0.05 was considered statistically significant without adjusting for multiple comparisons.

Approval was obtained from the institutional review board, Human Research Protections Office, at the University of New Mexico Health Sciences Center . Figure 1 , 71 subjects completed the baseline survey (which collects information for the traditional pretest evaluation), and 37 of these completed the 12-month survey (which collects information for the retrospective pretest and posttest evaluation). Another 10 subjects joined the 12-month survey but did not complete the baseline survey. A total of 47 participants were therefore available for analyses. As shown in Table 1 , most respondents reported caring for miners for 10 years or fewer (55.3%) and caring for at least a 40% proportion of miners living in rural areas (59.6%); 53.2% of subjects were existing participants. Clinical professional groups constituted the largest group of subjects (74.4%), with the largest subgroup being clinicians. Despite a high level of reported professional satisfaction, a significant minority described professional isolation.

Using the retrospective pre-posttest method, with both pretest and posttest data obtained at the 12-month study time point, subjects reported significant improvements in 10 of 14 items (P < 0.05, positive average change scores) and a significant decline in 1 of 14 items (with respect to their ability to 2. Self-efficacy items at baseline (pretest), estimated retrospectively (retrospective pretest), and at 12 months (posttest). Error bars are 95% confidence intervals. The symbols inside the left graph margin indicate results from nonparametric Wilcoxon signed rank tests for change measured retrospectively (retrospective pre-posttest; n = 47) and from baseline (traditional pre-posttest; n = 37) using paired measurements ( -P ≥ 0.05, *P < 0.05, **P < 0.01, and ***P < 0.001). Raw data for this figure are provided in Table 2 . "Diagnose" indicates the ability to diagnose common health conditions in miners; "manage" indicates the ability to help manage common health conditions in miners; "interpret PF tests" indicates the ability to interpret pulmonary function (PF) test results; "assess PF test quality" indicates the ability to assess the quality of the PF test; "interpret BG tests" indicates the ability to interpret arterial blood-gas (BG) test results; "interpret X-rays" indicates the ability to interpret chest radiograph reports using the International Classification of Radiographs of Pneumoconiosis (i.e., B reads); "determine eligibility" indicates the ability to determine eligibility for compensation under specific miners' compensation programs; "advocate/navigate" indicates the ability to advocate for your patient/client to help them navigate the compensation process; "collect information" indicates the ability to collect information required under the miners' compensation programs; "demonstrate empathy" indicates the ability to demonstrate empathy toward miners with work-related diseases; "identify barriers" indicates the ability to identify social, linguistic, cultural, economic, and educational barriers to care for miners; "community/regional expert" indicates the ability to serve as the miners' expert in your community/region; "collaborate" indicates the ability to collaborate with and educate other team members about miners' diseases; "make referrals" indicates the ability to refer patients with diseases to appropriate experts when you do not possess the relevant expertise; "Avg clinical skills" indicates the average (Avg) 6-item clinical-skills-domain score (diagnose, manage, interpret PF tests, interpret BG tests, and interpret X-rays); "Avg medicolegal skills" indicates the Avg 3-item medicolegal-skillsdomain score (determine eligibility, advocate/navigate, and collect information); "Avg soft skills" indicates the Avg 5-item soft-skills-domain score (demonstrate empathy, identify barriers, community/regional expert, collaborate, and make referrals); and "Avg total score" indicates the Avg 14-item total score. interpret pulmonary function test results; P < 0.001, negative average change scores) since their start dates in the program ( Table 2 and Figure 2 ). Subjects also reported significant improvement with respect to their scores for each of the three domains of clinical skills (P < 0.001), medicolegal skills (P = 0.04), and soft skills (P = 0.01) and for the 14-item total score (P = 0.002). As compared with fresh participants, existing participants demonstrated significantly greater improvement in their self-reported ability to diagnose health conditions (i.e., clinical-skill item 1; estimate, 0.78; 95% Table 3 ).

In the alternate analysis of 37 subjects, using the traditional pre-posttest method, with pretest and posttest data obtained at the 0-and 12-month study time points, respectively, a significant improvement in 3 of 14 items (P < 0.05) and a decline in 1 item (with respect to their ability to interpret pulmonary function test results; P = 0.04) was noted over the 12-month study timeframe (Table 2) . Subjects showed no significant improvement in any of the three domain scores or their total score over this timeframe. As compared with the nonclinical professional groups, clinical professional groups demonstrated significantly greater improvement in their self-reported ability to determine eligibility for compensation (i.e., medicolegal-skill item 1; estimate, 1.67; 95% CI, 0.25-3.08; P = 0.03), their self-reported ability to advocate for the patient/client to help them navigate the compensation process Definition of abbreviation: SD = standard deviation. The change in self-efficacy items, separately calculated since the subject start date in the program, was determined using the retrospective pre-posttest method with both pretest and posttest data obtained at the 12-month study time point. The change in self-efficacy items over a 12-month study timeframe was calculated using the traditional pre-posttest method, with pretest and posttest data obtained at the 0-and 12-month study time points, respectively. Improved item scores are indicated by positive change scores, and negative change scores indicate a decline in item scores. Absolute values for effect sizes for statistically significant differences ranged from 0.31 to 0.59, indicating only small-to-medium clinically meaningful changes (34) . The data for the change in scores for those participating for both types of testing strategies, limited to 37 subjects who had both 0-and 12-month data measured, showed similar results, as provided in Table E5 . Table 4 ).

Telementoring multidisciplinary professional groups in the care of miners improved participants' self-efficacy with respect to clinical, medicolegal, and soft skills. The retrospective pre-posttest method used since the participant start date with the program demonstrated improvement in a greater number of self-efficacy items and aggregate scores than the traditional pretest and posttest method over a 12-month study period. Existing participants and clinical professional groups demonstrated greater improvement in selected items than fresh participants and nonclinical professional groups, respectively. Our study demonstrates that the ECHO model can be successfully applied to professionals providing complex multidisciplinary care to miners.

Mining is a key economic activity in the rural United States, with potential for health hazards. With the United States having 28% of the world's recoverable coal reserves, mining employment in the United States is likely to remain significant for many years, despite recent mine closures (20) . It is anticipated that pneumoconiosis For definition of abbreviation, see Table 2 .

Positive change scores indicate improved item scores, and negative change scores indicate a decline in item scores. Table 4 . Subgroup analysis of change in self-efficacy items over a 12-month study timeframe (using the traditional pre-posttest data method with pretest and posttest data obtained at 0-and 12-mo study time points, respectively) will continue to be seen in U.S. miners for several decades because of the long latency period of the disease. Miners constitute an underserved, isolated, medically vulnerable, and often underinsured rural population (2) . In the southwestern United States, miners are predominantly Hispanic or American Indian high-school dropouts (2) . The key challenges for caring for miners are the lack of adequately trained rural providers, perceived isolation among existing rural providers, and the complexity of the miners' compensation systems (based on our preprogram launch needs assessment, A. Sood and colleagues, unpublished results). Addressing these challenges is a first step in fighting the emerging epidemic of pneumoconiosis but requires access to multidisciplinary expertise and training. The ECHO model leverages the power of information technology and case-based learning to provide state-of-the-art training and mentorship for such professionals in underserved areas. The Miners' Wellness TeleECHO Program has successfully recruited stakeholder groups from mining communities in pneumoconiosismortality hot-spot regions of rural Appalachia and the Mountain West. The target audience for the TeleECHO Program includes both clinical and nonclinical professionals who diagnose, treat, manage, and assist miners with pneumoconiosis. Evidence from our needs assessment demonstrates that treating pneumoconiosis is a complex process involving a difficult-to-navigate miners' compensation system in the United States. Our multidisciplinary telementoring approach is essential to enable both clinical and nonclinical professionals to collaboratively develop comprehensive plans for providing the best possible care to miners with pneumoconiosis. For definition of abbreviation, see Table 2 .

Positive change scores indicate improved item scores, and negative change scores indicate a decline in item scores.

We found different results using our two analytic strategies: retrospective pre-posttest assessment since participant start date with the program versus traditional pretest and posttest assessment over a 12-month study period. The effect sizes tended to be greater, and changes were more likely to be statistically significantly different, using the former versus the latter approach, as also described by prior studies of ECHO outcomes (18, 19) . By presenting the pre-and postassessments simultaneously in the retrospective pre-post design, respondents are forced to rate their status from the same frame of reference, which can help control for "response-shift bias" (21) . This bias occurs when respondents use one frame of reference at the pretest but use a different frame of reference at the posttest, rendering it difficult to compare responses and resulting in erroneous inferences about program effectiveness. On the other hand, because the retrospective pre-post design queries participants "after" they have received the intervention, the participants may intentionally misrepresent the degree of impact they received from the intervention. Furthermore, given the possibility of recall bias in a retrospective pre-post design, this study design may replace one set of biases (vis-à-vis the traditional pre-posttest study design) with another (22).

Despite improvement in multiple selfefficacy parameters, our analysis indicates a decline in self-efficacy with respect to interpreting pulmonary function tests. This may relate to the confusion among rural professionals from multiple pulmonary-function-test updates by professional organizations since 2017 (3, (23) (24) (25) (26) . This also implies a need for greater attention in the future to this competency gap identified by our telementoring program. The fact that existing participants show greater improvement than fresh participants indicates that there may be a dose effect of telementoring on change in self-efficacy. This conclusion is supported by the finding that additional adjustment for participation frequency reduced the magnitude of differences between fresh and existing participants (see Figure E1 and Table E6 ). In addition, greater improvement in outcomes among participants belonging to the clinical versus nonclinical professional groups may reflect the disproportionate emphasis of clinical didactics and case discussions in the program (see Table E3 ). This implies a need for greater attention to nonclinical topics by our multidisciplinary program.

A strength of the current study is that our ECHO intervention is located in a rural and poor but mining-intense state, designated by the U.S. Health Resources and Services Administration as a medically underserved area. We believe that this increases the generalizability of the study findings to other rural mining regions. In addition, we tested the feasibility of applying a model of healthcare delivery that is already well established in other diseases, such as hepatitis C, by reducing variation in processes of care and sharing best practices (12) (13) (14) (15) . It is likely that this model of healthcare delivery may be similarly useful in the management of other lung diseases requiring multidisciplinary discussion, such as interstitial lung diseases (27) . Because the ECHO model has been adopted nationally and globally to improve access to care in a number of disciplines, there is substantial infrastructure already in place to allow for rapid scaling of the Miners' Wellness TeleECHO model. This telementoring approach was particularly beneficial during the coronavirus disease (COVID-19) pandemic, during which multiple face-to-face professional educational programs were canceled, resulting in its recognition as a rural COVID-19 innovation by the U.S. Health Resources and Services Administration's Rural Health Information Hub. The American Thoracic Society also recognized the program as a 2019 innovation in fellowship education.

There are also limitations to the current study. We are unable to correlate improvement in professional self-efficacy with patient outcomes. The literature, however, indicates that professional self-efficacy influences behavior or performance and improves patient outcomes (28) . For example, self-efficacy is a good predictor of asthma-guideline adherence among primary care clinicians (29) and of clinical performance of student physician assistants (30) . Investigators report positive findings using randomized controlled designs to test the efficacy of electronic-learning interventions on novice surgeons' surgical performance of laparoscopic cholecystectomy (31) and nurses' ability to detect child abuse in an emergency department (32), respectively. A listing of qualitative changes that our ECHO participants reported they were going to make in their practice, obtained as part of a CME survey requested at the end of each TeleECHO session, are provided in Table E2 (with other quantitative outcomes provided in Table  E1 ). A small sample size and multiple comparisons in the current study raise the possibility of type I error. High-risk individuals who drop out of the program or do not volunteer to participate in the study do not provide information in the estimation of the program effects, thus introducing an element of potential participation bias. The self-efficacy instrument was not fully validated in the current study. Standardized Cronbach α-coefficient values for the instrument, computed from the baseline measurement, however, showed high levels of internal consistency (values ≥ 0.85 for all items and each of the three domains).

This study is the first in a stepwise approach to determine the benefit of participating in a multidisciplinary telementoring intervention by improving participant self-efficacy in delivering high-quality care to miners with complex mining-related diseases. The acknowledgment of provider self-efficacy as a common barrier to optimal care of complex diseases (33) and as a predictor for provider behavior or performance and for patient outcomes (28) in the literature underscores the relevance of employing the TeleECHO intervention in professions caring for miners in medically underserved regions. Although future studies are required to determine any improvement in participant behavior or performance or miner outcomes, our study finding represents a potential solution to a growing access-to-care gap for miners with pneumoconiosis. This provides the rationale for using systems that are already in place for rapid scaling of the Miners' Wellness TeleECHO Program at other institutions nationally and globally.

Radiographic disease progression in contemporary US coal miners with progressive massive fibrosis

An innovative approach to enhancing access to medical screening for miners using a mobile clinic with telemedicine capability

Trends and patterns of differences in chronic respiratory disease mortality among US counties

A virtual community-ofpractice approach by rural stakeholders in managing pneumoconiosis in the USA: a cross-sectional analysis

Communities of practice: learning, meaning and identity

Self-efficacy: toward a unifying theory of behavioral change

Why don't physicians follow clinical practice guidelines? A framework for improvement

Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities

Deliberate practice and acquisition of expert performance: a general overview

Social cognitive theory: an agentic perspective

How we learn: communities of practice-the social fabric of a learning organization

Expanding access to hepatitis C virus treatment: Extension for Community Healthcare Outcomes (ECHO) project: disruptive innovation in specialty care

Endo echo improves primary care provider and community health worker self-efficacy in complex diabetes management in medically underserved communities

Taribavirin in the treatment of hepatitis C

Outcomes of treatment for hepatitis C virus infection by primary care providers

Further validation of the multiple sclerosis self-efficacy scale

Project ECHO evaluation 101: a practical guide for evaluating your program

Supporting and improving community health services-a prospective evaluation of ECHO technology in community palliative care nursing teams

Evaluation of Project ECHO (Extension for Community Healthcare Outcomes) Northern Ireland programme 2015-6

Coal mine dust lung disease: new lessons from old exposure

Comparing traditional versus retrospective pre-/post-assessment in an interdisciplinary leadership training program

Using a retrospective pretest instead of a conventional pretest is replacing biases: a qualitative study of cognitive processes underlying responses to thentest items

ATS Committee on Proficiency Standards for Pulmonary Function Laboratories. Recommendations for a standardized pulmonary function report: an official American Thoracic Society technical statement

ERS/ATS standards for single-breath carbon monoxide uptake in the lung

Executive summary: 2017 ERS/ATS standards for single-breath carbon monoxide uptake in the lung

Standardization of spirometry 2019 update: an official American Thoracic Society and European Respiratory Society technical statement

Role of a regional multidisciplinary conference in the diagnosis of interstitial lung disease

The effectiveness of internet-based e-learning on clinician behavior and patient outcomes: a systematic review protocol

Clinician agreement, self-efficacy, and adherence with the guidelines for the diagnosis and management of asthma

The relationship between self-efficacy and student physician assistant clinical performance

Multimediabased training on Internet platforms improves surgical performance: a randomized controlled trial

Successful e-learning programme on the detection of child abuse in emergency departments: a randomised controlled trial

Provider selfefficacy and the screening of adolescents for risky health behaviors

Statistical power analysis for the behavioral sciences

Telementoring in the Care of Miners

Author disclosures are available with the text of this article at www.atsjournals.org.