key: cord-1039746-cdxl9jtn
authors: Coon, Jodi C.; Bush, Helena; Rapp, John T.
title: Eight Months of Telehealth for a State-Funded Project in Foster Care and Related Services: Progress Made and Lessons Learned
date: 2022-03-16
journal: Behav Anal Pract
DOI: 10.1007/s40617-022-00682-z
sha: 8c3239c1744227c411d905520464911d2340e703
doc_id: 1039746
cord_uid: cdxl9jtn

In response to the COVID-19 pandemic, many behavior analysts and other health professionals modified their services for delivery via telehealth modalities. The transition to telehealth is especially important for providers working with foster youth who exhibit challenging behavior because these youth often move to another placement due to such behaviors. The primary objective of this article was to evaluate the extent to which service indicators for a state-funded team working with foster youth changed after the service delivery model changed from in-person to telehealth services. In particular, we evaluated changes in monthly count of client contacts, appointments, intakes, closed cases, and medication reviews. The secondary objective was to outline potential benefits and environmental barriers encountered by the team and to integrate our findings with the literature on behavior-analytic services provided via telehealth. Overall, results show that we maintained service quality with a broad range of behavioral interventions and increased overall client appointments. Given these outcomes, our team may continue to provide behavioral services via telehealth after the COVID-19 pandemic.

In response to the COVID-19 pandemic, many behavior analysts and other health professionals modified the delivery of their services by using telehealth modalities. Broadly defined, telehealth is a service delivery modality that allows practitioners to provide services and interventions from a distance through telecommunication technologies (Leblanc et al., 2020) . An alternative to inperson appointments, telehealth allows practitioners to conduct appointments either synchronously (live two-way video or audio interactions between therapist and participant) or asynchronously (transmission of participant recordings or data that are later reviewed by a therapist; Council of Autism Service Providers, 2020). Throughout the pandemic, many behavioral and mental health practitioners, including board certified and licensed behavior analysts 1 (LBAs), have provided services exclusively via telehealth. Though not all practitioners made a transition to exclusively telehealth, many have provided a hybrid of services combining telehealth and in-person services. By increasing the use of telehealth, professionals are able to decrease their contact with others and thus decrease the chances of both contracting and spreading the coronavirus.

Providing behavior analytic services via telehealth is not novel. In fact, researchers have already explored telehealth as a method to provide behavioral services to individuals in rural settings and other areas with limited access to behavior analysts. Ferguson et al. (2019) identified nearly 30 studies that evaluated telehealth behavior analytic services to individuals with autism spectrum disorder (ASD). In their analysis, Ferguson et al. found that 43% of reviewed studies used functional analysis (FA) and functional communication training (FCT) and 36% of the studies included techniques derived from incidental or naturalistic teaching procedures. The remaining studies included procedures for teaching caregivers to implement preference assessments, evaluating behavior support strategies, or providing a comprehensive overview of behavior analysis. Likewise, Tomlinson et al. (2018) used telehealth modalities to teach caregivers to implement behavior analytic interventions.

Despite the technology requirements of telehealth, this service delivery model may afford numerous benefits to practitioners. As projected by LeBlanc et al. (2020) , the benefits will likely lead to continued use of telehealth even after environmental constraints subside. As previously noted, one primary benefit of telehealth is increased access to behavior analytic services, especially for individuals living in rural areas (Wacker, Lee, Dalmau, Kopelman, Lindgren, Kuhle, Pelzel, Dyson, et al., 2013a; Wacker, Lee, Dalmau, Kopelman, Lindgren, Kuhle, Pelzel, & Waldron, 2013b) and countries with limited behavior analysts (Tsami et al., 2019) . In addition to expanding the range of service recipients, telehealth may decrease the time participants spend on wait lists . Lindgren et al. (2016) compared in-home therapy, clinic telehealth, and home telehealth, and found that the mean total cost per child to complete assessment and treatment was significantly lower for telehealthbased service models. Additional benefits include increased access to various training formats (Ferguson et al., 2019; Neely et al., 2017; Tomlinson et al., 2018) , more opportunities to assess generalization and maintenance of skills in natural settings, and less intrusive observations (LeBlanc et al., 2020) . Although evidence supports using telehealth to teach caregivers to implement various behavioral assessments and interventions (Unholz-Bowden et al., 2020) , there is limited research that evaluates the effectiveness of using telehealth to teach various other skills. Moreover, there are limited comparisons of in-person and telehealth services.

The U.S. Congress passed the Child and Family Services Improvement and Innovation Act (2011) in response to growing national concern over the high percentage of foster youth who received psychotropic medication. This public law requires each state to develop plans to provide oversight and monitoring of psychotropic medications prescribed to foster youth. The Alabama Department of Human Resources (ALDHR) Family Services division developed the Alabama Psychiatric Medication Review Team (APRMT) to address this requirement (Luna et al., 2018 (Luna et al., , 2020 . ALDHR initially developed the team to provide services to foster parents and foster children; however, in 2019 ALDHR requested that the APMRT expand services to include home preservation cases (e.g., biological parents, adoptive parents, and their children). The APMRT comprises five LBAs (four LBAs deliver behavioral services), a child and adolescent psychiatrist, a psychiatric mental-health nurse practitioner, a psychopharmacologist, and multiple graduate students pursing board certification in behavior analysis. In part, ALDHR charged the APMRT with (1) developing procedures for tracking and decreasing the use of psychotropic medications as treatment for problem behavior, (2) replacing psychotropic medications with behavioral intervention, and (3) tracking multiple service indicator measures, which the APMRT reports monthly to ALDHR. Because prescribers typically treated problem behavior with one or more psychotropic medications (Zito et al., 2008 (Zito et al., , 2021 , the primary aim of most interventions was reducing each child's challenging behavior. As such, the first step in service delivery involves assessing target behavior(s), developing an individualized behavior plan, and training stakeholders to implement the plan. As a part of this process, LBAs address skill deficits that contribute to the target behavior, such as communication training and tolerating events.

Prior to March 2020, members of the APMRT exclusively provided in-person training sessions in the homes of foster and biological parents. Beginning in April 2020, the team shifted to providing behavioral and medication review services (described below) exclusively via telehealth using synchronous telehealth methods such as phone calls and web conferencing via secure platforms. The primary purpose of this discussion article is to evaluate possible changes in several service delivery indicators following our transition from in-person to telehealth service delivery. The secondary purpose of this article is to highlight the telehealth behavioral services provided by the APMRT, identify some of the obstacles encountered by the APMRT throughout telehealth service delivery, and relate our proposed solutions to those obstacles and the literature.

Participants were (1) youth aged 2-18 years who had an open case with ALDHR (e.g., foster care, at-risk youth, adoptive placement) and (2) the respective caregivers (i.e., adoptive parents, foster parents, or biological parents). All participants had been referred for or received behavioral services, thirdparty medication review services, or both from the APMRT. 2 Medication reviews were required for every participant who had been prescribed psychotropic medication. As a part of this process, either the psychiatric nurse practitioner or the child psychiatrist (hereafter, medical professionals) determined the extent to which medications were (1) prescribed within the FDA's dosage guidelines, (2) appropriate for the participant's age and size, (3) appropriate for the participant's diagnoses, and (4) likely to interact with other medication in the participant's regimen. The medical professional wrote a formal letter and discussed it with the respective participant's LBA. Thereafter, the LBA explained the letter to the participant's caregiver and provided additional training regarding appropriate and effective communication about the participant's medication with the participant's prescriber (i.e., medication advocacy training). In short, only the team's two medical professionals provided written statements about the appropriateness of a given participant's psychotropic medication regimen; however, the consulting LBA, who had regular contact with the caregiver, typically delivered those recommendations to the caregiver. Prescribers included pediatricians, family practice physicians, nurse practitioners, and psychiatrists. The LBA provided one copy of the medication review to the primary stakeholder and another to relevant case personnel (e.g., the participant's social worker). The team members conducted medication reviews at various points throughout services (tis typically occurred at intake, when medication changed, and following multiple weeks with improved behavioral symptoms) to evaluate when decreasing psychotropic medications was appropriate. At the end of the process, the participant's prescriber made decisions regarding participants' medication. It should be noted that LBAs provided medication review and medication advocacy training 3 in conjunction with behavior-analytic services.

During the analysis period for this study, the APMRT completed 28 intakes, progressed to the assessment and intervention phase for 19 of those participants and referred the remaining 9 participants to an LBA who could provide in-person services. 4 In addition, 10 participants chose to make the transition to services via telehealth rather than be placed on an in-home waitlist or be referred to another LBA. In sum, APMRT provided behavioral services for 29 participants throughout the analysis period. Ten (34%) participants received in-home services prior to telehealth, and 19 (66%) participants received only telehealth services. See Tables 1, 2 and 3 for participant characteristics, diagnoses, and referral concerns, respectively. Figure 1 shows the flow of services provided by the APMRT. Although the APMRT abruptly changed to a telehealth service-delivery model, we continued to implement similar assessments as when we provided in-home services. For three clients with more severe challenging behavior, LBAs completed functional analyses (FA). To this end, studies have demonstrated the utility of remote FAs (e.g., Wacker, Lee, Dalmau, Kopelman, Lindgren, Kuhle, Pelzel, Dyson, et al., 2013a; Wacker, Lee, Dalmau, Kopelman, Lindgren, Kuhle, Pelzel, & Waldron, 2013b) . In the majority of studies, behavior analysts remotely supervised either a trained professional or a caregiver on conducting FAs in participants' homes. For participants receiving services from the APMRT, foster or biological parents implemented all telehealth FAs while an LBA supervised remotely. Given that the target behaviors occurred primarily in the home, parent-implemented FAs reliably evoked participants' target behaviors. In particular, one participant's challenging behavior was maintained by escape from demands, one participant's behavior was maintained by access to attention, and one participant's behavior was maintained by escape from demands and access to attention and tangibles. Team members also used telehealth modalities to conduct preference assessments, assess idiosyncratic target contexts such as tolerating losing games, and assess and treat various academic (e.g., reading, writing, and math) and social skill deficits. When providing in-home services, the LBA conducted these assessments using various tangible stimuli. Due to the constraints of telehealth, the LBAs used Microsoft PowerPoint© for conducting preference assessments (typically without contingent access to selected items) and assessing various skill deficits.

Similar to the success in continuing adequate behavioral assessments, LBAs targeted various behaviors and skills by adapting interventions regularly implemented during inperson services. LeBlanc et al. (2020) advised researchers to continue demonstrating the diversity of procedures that are possible via telehealth. Table 4 provides a comprehensive list of the interventions the LBAs implemented during the 8 months of telehealth services, and includes a brief description of each intervention and the total count of implementations. Of the intervention types listed, all nine were common strategies used by LBAs during in-person services. However, LBAs had not previously implemented programs such as fluency training and responding to listener disinterest. This list exemplifies the variety of interventions that our team either adapted or developed for services via telehealth.

Our team experienced some of the aforementioned benefits of telehealth services. For LBAs, benefits included increased appointments with families, decreased travel time, increased time for indirect and administrative tasks, and increased training opportunities for graduate students. It should be noted that the manner in which the APMRT provides services likely differs from clinic settings. In particular, APMRT LBAs are responsible for all indirect and administrative tasks, participant meetings, and intervention implementation, and do not utilize registered behavior technicians.

Before telehealth services, each LBA spent an average of 7 hr (18%) of each workweek traveling. The switch to telehealth modalities eliminated travel time for LBAs. In turn, LBAs were able to schedule more frequent participant appointments and reallocate time to other participant-related tasks such as program development and professional development. The average number of appointments completed per LBA increased from 12 during the first month 5 of telehealth services to 26 during the last month of analysis, representing a 217% increase in 8 months. Because caregivers were the sole behavior change agents throughout telehealth, LBAs increased the number of appointments dedicated to training caregivers on how to conduct behavior analytic procedures such as how to conduct FAs, delivering discriminative stimuli, and delivering consequences during interventions. For the last 6 months of in-person services, the cost of LBAs' travel reimbursement was $7,548.00. In the first 8 months of telehealth services, travel reimbursement decreased to $0.00. Participants benefited from greater flexibility in scheduling, more frequent appointments, increased parent training opportunities, and a shortened wait list, which decreased by 44% during telehealth services. In addition, time saved from decreased travel allowed LBAs more time to develop training protocols for graduate students and caregivers, help caregivers make the transition to telehealth services, and make procedural adjustments.

After we altered our service delivery to a telehealth format, we continued to monitor multiple service indicators (described below). We subsequently compared the outcomes for telehealth service delivery to traditional in-person service delivery. We used visual analysis in conjunction with the dual criterion (DC) method (Fisher et al., 2003) to detect differences between service indicators (i.e., dependent variables) across the two service modalities. The DC method requires projecting regression and mean lines from the A phase (in-person services) across the ensuing B phase (telehealth services). We depicted each dependent variable in a separate graph containing six data points (months) in the A phase and eight data points in the B phase, which permits optimal use of the DC analysis (Lanovaz et al., 2017) .

According to the cutoff criteria described in Fisher et al. (2003, p. 399) , at least seven data points are required to fall below both the regression and mean lines in order to claim there was a statistically significant decrease in the dependent variable in the B phase relative to the A phase (or vice versa for a significant increase). Figure 2 shows the count of participants contacted during the last 6 months of in-person services as well as the 8 months after beginning telehealth only services. Visual inspection shows that there was no change in the count of participants contacted, indicating that the APMRT was able to provide behavioral services to a similar number of participants during in-person services and telehealth services. The DC analysis confirmed that there was not a statistically significant difference between participants contacted before and after telehealth began. Figure 3 shows the count of intakes for behavioral services completed across months. Visual inspection suggests there was no change between inperson and telehealth services, and results of the DC analysis results indicate there was not a statistically significant difference between the two phases. In sum, the transition to telehealth did not significantly alter the number of participants contacted. Figure 4 depicts the count of behavioral services cases closed across months for the last 6 months of in-person services and 8 months of telehealth services. LBAs typically closed cases (1) after they addressed the initial referral concerns for the respective child or adolescent, (2) when the foster or biological parent failed to respond to three or more Table 4 Interventions completed using telehealth DRA differential reinforcement of alternative behavior, FCT Functional Communication Training, DRL differential reinforcement of lower rates of behavior, BIP behavior intervention plan a Two participants moved in the middle of services, and one participant subsequently met mastery criteria when in-person services resumed b Data for two participants could not be confirmed due to the residential facility's COVID-19 restrictions 

Number of participants contacted across months. Note. Some participants were contacted but did not receive services for various reasons (e.g., client moved, caregiver did not want telehealth services). In addition, for four of the months, the count of participants contacted is higher than 25 (the number of total participants who received telehealth services). This occurred when an LBA contacted a case worker or caregiver about a participant, but behavioral services were not delivered (e.g., meeting with DHR, caregiver request services be terminated) Fig. 4 Number of behavioral services cases closed across months contacts, or (3) when a foster child was moved to a placement outside the project catchment areas (this only happened during in-person services). Visual analysis shows that after beginning telehealth services, the number of closed cases stabilized, most notably starting in month 11. Although the majority of data points fall below the mean line, at least seven data points would need to fall below both the regression and mean lines in order to claim that the count of cases closed was significantly lower during telehealth service delivery. Thus, based on both visual and DC analyses, making a transition to telehealth service delivery did not significantly alter the number of behavioral cases closed. In a study conducted in Wyoming, Hilt et al. (2015) found that telemedicine psychiatric consultation, remote medication reviews, and voluntary phone-based consultations with a child and adolescent psychiatrist significantly decreased the amount of psychotropic medications prescribed to children 5 years of age or younger by 42%. In this vein, Fig. 5 depicts the number of medication reviews 6 completed across each month. As previously noted, medication reviews are a unique service of the APMRT and involve a review of a participant's medications, but does not include behavior services. There was a high number of medication reviews during the 2 months before and after telehealth began, but visual analysis shows the number of reviews stabilized and remained stable for the last 5 months of analysis. The results of the DC analysis indicate there was not a statistically significant difference in the number of medication reviews completed across the two phases. In sum, making the transition to telehealth services did not significantly change the number of medication reviews the APMRT completed. Figure 6 shows the number of appointments completed during the last 6 months of in-person services and the 8 months after beginning telehealth-only services. Visual inspection suggests that there was a clear increase in the number of appointments completed from the in-person phase to the telehealth phase. Results from the DC analysis indicate that the increase was statistically significant. 7 In sum, the transition to telehealth services resulted in a significantly higher number of appointments. This change appeared to reflect increased appointments with a consistent number of clients rather than an increase in the number of clients contemporaneously served (i.e., the LBAs' caseloads remained stable). The increased number of appointments was likely due to the increased caregiver training that was required throughout telehealth. As previously noted, caregivers required multiple training sessions for conducting assessments and implementing interventions. As such, participants did not progress through the model more quickly; rather, participants' caregivers received more appointments to facilitate the transition to telehealth. Nevertheless, the time span covered in this article did not allow for a thorough comparison of "days receiving services" between the two modalities. Taken together with the decrease in the amount of time spent and cost of travelling, the increase in client appointments suggests that telehealth may require additional parent training but could be a more efficient service delivery model than in-person services for appropriate cases.

Given our shift to telehealth service, we were concerned that we had inadvertently served fewer families with low socioeconomic status (SES) due to the requirements of receiving telehealth services (e.g., access to either internet services or smart phones), families with numerous individuals in their home who competed for computer access, or both. To address these questions, we compared the two groups for differences in education level of the parents in the home 8 (foster or biological) and the number of individuals The APMRT is in the process of evaluating the extent to which medication reviews influence prescribers' behavior. 7 Data recorded for three clients in the A phase were incomplete. Thus, the counts could have been slightly higher. 8 We excluded individuals who resided in facilities from this analysis.

residing in the house (LBAs collected this information during the initial intake meeting). For the former question, we used education level as a proxy for SES and categorized each foster or biological parent's education level into one of the following: (1) less than a high school degree, (2) high school degree, (3) Associate degree, or (4) Bachelor's degree or higher. Results of a Chi-square test for independence indicated there was no significant association between group and level of education, χ 2 (4, n = 46) = 1.36, p = .714. With regard to the latter question, the mean number of individuals in households for in-person (M = 4.43, range: 2-7) and telehealth (M = 4.3, range: 2-9) services also did not appear to differ (no statistical analysis was needed given the appearance of equality). Thus, these results suggest our shift from in-person to telehealth service delivery did not selectout families with either lower educational backgrounds or more individuals in their household. Nevertheless, in general LBAs did not provide behavior services via telehealth modalities for individuals who displayed severe aggression or property destruction.

Although telehealth modalities offer some potential benefits, practitioners must overcome a variety of barriers to ensure the delivery of quality services. To this end, researchers have highlighted ethical considerations (Pollard et al., 2017; Romani & Schieltz, 2017) and best practice recommendations for telehealth services (Council for Autism Service Providers, 2020; Rios et al., 2018) . Lee et al. (2015) also created a guide for troubleshooting telehealth technology issues that arise during home-based assessments and treatments. Lerman et al. (2020) outlined specific challenges that might emerge related to technical issues, remote viewing, disruption in the participant's environment, participant and caregiver behavior, and site issues. Lerman et al. recommended informing practitioners about potential challenges because it may increase practitioners' knowledge and use of preventative strategies. To expand on the challenges outlined in the literature, Table 5 provides a list of barriers the APMRT encountered during our transition to telehealth service delivery, as well as the solutions we employed. Team members readily transferred some services such as intake interviews and medication reviews to a telehealth service modality. However, the delivery of subsequent services required more consideration. For all participants, LBAs had to modify how they built rapport, completed observations, and conducted parent training. When in homes, LBAs often built rapport by participating in activities with the participant. After the transition to telehealth, the LBAs were limited to virtual rapport-building activities such as playing games online and watching preferred videos with the participant. Another transition that required an extensive amount of consideration was how to observe the target behaviors and provide parent training. During in-person services, LBAs could easily move throughout the home to observe the participant. After the transition to telehealth, LBAs were limited by the placement of the phone or computer and were not as easily able to observe the behaviors in various locations of the home. To address this challenge, some observations were completed via a secure platform on a cell phone rather than a computer to better allow the LBA to observe the participant's behavior in multiple locations. In addition, the LBA turned his or her camera off and provided instructions to the caregiver through headphones for participants who were easily distracted by computers.

Finally, parent training had to be altered during telehealth service delivery because LBAs were often not able to model strategies with the participant. One noteworthy solution we employed was utilizing video recordings of the participants taken during the assessment period. The LBA recorded baseline observations and used clips from those recordings throughout the parent training process. Because the LBAs used previously recorded behavioral episodes, caregiver training was expedited in some cases because rehearsing appropriate responses was not contingent on the participant displaying target behaviors in later appointments. Instead, the LBA used the recorded behavioral episodes to (1) specify the target behavior, (2) instruct and model (for the caregiver) how to respond when the target behavior occurred, and (3) verbally rehearsed with the caregiver how to respond. Once the instruction, modeling and rehearsal phases were complete, the LBA provided frequent feedback on the caregiver's implementation of the behavior plan in real time. In sum, the shift to telehealth required more planning at the beginning of parent training, but likely resulted in a more efficient parent training process.

Overall, we experienced some previously identified barriers, but we have outlined additional noteworthy solutions. For example, we used automated features such as Microsoft Flow® to send reminders about appointments or reminders to fill out data sheets. We also modified our data collection so that caregivers received a link on their cell phone each night from Microsoft Flow® to prompt them to enter data for the day. In some cases, LBAs also taught participants necessary telehealth behaviors (e.g., keeping body in front of the screen, requesting breaks) if antecedent environmental modifications did not suffice. Nine families 9 opted to forgo telehealth services, which likely contributed to the decreased waitlist time for other families. It should be noted that we frequently modified training materials to suit the caregiver's preferences and needs. By increasing caregiver choice among materials, instruction delivery, communication modes, and service provision, we were able to provide behavior analytic services throughout the COVID-19 pandemic for all but nine participants. By examining Table 5 , practitioners may be able to identify barriers similar to theirs and generalize our solutions to their respective settings and challenges.

We conducted a brief survey to assess the LBAs' satisfaction with the transition to telehealth services. Each LBA on the team completed the survey, and survey items and results can be found in Table 6 . Overall, LBAs were satisfied with the transition to telehealth services and would be willing to provide telehealth services in the future. Three LBAs said they preferred a hybrid of in-person and telehealth service delivery, and one LBA stated they preferred in-person service delivery.

The primary purpose of this article was to highlight behavioral assessments and interventions that a state-funded team conducted and implemented using telehealth and provide a guide for practitioners to follow when making similar transitions or searching for interventions to be delivered remotely. The secondary purpose of this article was to outline various telehealth benefits and environmental barriers that we faced and propose solutions to such barriers. Barriers such as a placement disruption (i.e., a foster parent requesting the child be removed from the home) are unique to foster youth, but such a barrier should prompt practitioners to further consider potential challenges that may arise within their respective population. Furthermore, our team provides services exclusively to youth involved with the child welfare system, an underresearched and underserved population. Youth involved with the child welfare system often move frequently, and the chances of moving increases with the presence of challenging behaviors (Leathers, 2002; Van Holen et al., 2017) . As the acceptance and research of applied behavior analysis in novel populations increases, service delivery formats such as telehealth will extend the reach of providers to deliver services to families who otherwise would not be able to receive services. It should be noted that the majority of participants did not require instruction regarding appropriate telehealth behaviors (e.g., orientation to screen), had low levels of severe behaviors, and could communicate vocally. However, we recognize that many LBAs work with individuals who have deficits in these areas. This article contributes to the telehealth literature by providing evidence that certain indicators of service delivery did not degrade, and one important indicator improved, following our transition to a telehealth-only model. In particular, the number of client contacts, case closures, intakes, and medication reviews did not significantly decrease after the transition to telehealth. Moreover, appointments with families increased significantly during telehealth service delivery. This is noteworthy for two reasons. First, in addition to their own employment demands, many foster parents were required to home school (remotely) their foster children during the pandemic. The increased time with the foster children under an academic demand context gave rise to numerous problems with uncooperative and aggressive behavior. Without regular access to effective behavioral consultation, it is likely that many of these parents would have requested removal of their foster child due to problem behavior. Second, researchers have speculated that one reason why physicians overprescribe psychotropic medication for youth in foster care is they believe that other, nonpharmacological interventions are not available (Mayne et al., 2016) . By making the transition to telehealth service delivery, the APMRT ensured continuity of behavioral services. Moreover, we developed a medication advocacy (MA) protocol for which foster parents (1) received training from LBAs via telehealth modalities and (2) implemented MA components (e.g., showing graphs of daily ratings of behavior, asking medication-specific questions) during telehealth sessions with their foster child's prescriber.

The team also experienced a considerable cost savings by decreasing travel costs. Overall, these data support that behavior analysts utilizing a telehealth model can maintain service quality while experiencing additional improvements in some service delivery components. Practitioners may be interested in using these findings to support reimbursement requests for their telehealth services from funding agencies or when describing their activities in periodic reports. Although our team experienced many benefits of transitioning to a telehealth model and in general the LBAs were satisfied with the transition, it is crucial to consider the negative effects telehealth can bring on behavior analysts. For example, Jimenez-Gomez et al. (2021) found the transition to telehealth services increased burnout and decreased productivity of applied behavior analysts. As practitioners and companies consider creating telehealth or hybrid models of service delivery, it is imperative that they develop protocols to help mitigate the stressors associated with increased burnout when providing telehealth services.

The APMRT's experiences with telehealth service delivery gives rise to some potential avenues for future research. In line with the supposition made by LeBlanc et al. (2020) regarding generalization and maintenance of skills, caregivers were the primary change agents in most of our telehealth Table 6 LBA social validity survey a Question is anchored as following: 1: continued working on the same goals; 3: worked on some of the same goals; 5: were not able to continue working on the same goals b Question is anchored as following: 1: very little; 3: some; 5: a great deal c Question is anchored as following: 1: very dissatisfied; 3: neutral; 5: very satisfied d Question is anchored as following: 1: very little; 3: some; 5: a great deal

Average Rating Range

For the majority of clients, how much did switching to telehealth services alter your treatment plan? a 2.5 1 -3 To what extent did switching to telehealth increase the time you were able to dedicate to indirect client-related tasks? b 4 4 To what extent did switching to telehealth increase the time you were able to dedicate to administrative tasks? b 3.5 3 -4 How satisfied were you with the transition to telehealth? c 3.5 2 -5 To what extent did the transition to telehealth increase your job satisfaction? b 3 0 -5 If possible/necessary, how willing would you be to provide behavior analytic services via telehealth in the future? d 4.25 3 -5

interventions. During in-person services, LBAs planned for transfer of stimulus control, generalization, and maintenance by inserting additional intervention components. Because foster parents were the primary change agents at the outset of telehealth-only interventions, transfer of stimulus control procedures were often not necessary, in particular with respect to the treatment of challenging behaviors. Related to this, stimuli used throughout telehealth were directly from participants' homes, possibly creating stronger motivating operations and increasing the veracity of assessments and effectiveness of treatments. It seems plausible to predict that telehealth services can promote faster acquisition due to frequent contact with relevant discriminative stimuli, and that maintenance of acquired skills will be more durable due to the use of reinforcers found in participants' homes. Future studies should evaluate the speed of acquisition and the maintenance of skills for participants who receive telehealth only services compared to participants who receive in-person services only, or a hybrid of both. For example, datasets accumulated from comparable participants who received similar interventions that were delivered differently (i.e., telehealth versus in-person) could be compared. Researchers should also continue to explore the unique possibilities of behavior-analytic interventions provided via telehealth and collect data on the frequency of barriers encountered and success of solutions to those barriers. A large percentage of the telehealth literature pertain to the assessment and intervention of challenging behavior and communication training (LeBlanc et al., 2020) ; research is needed to develop the evidence base for the use of telehealth in other behavior analytic interventions. One practical way to continue such an investigation is to employ nonconcurrent multiple baseline designs after accumulating three or more data sets for a specific intervention (e.g., Coon & Rapp, 2018; Kratochwill et al., 2010) . In this manner, multiple practitioners across various sites could collaborate and rapidly accumulate numerous data sets.

As of 2020, 74.41% of board certified behavior analysts' primary area of emphasis is autism spectrum disorders (Behavior Analyst Certification Board [BACB], 2020), an increase from 54% in 2011 (BACB, 2011) . However, as behavior analysts continue to expand their work with incarcerated populations, geriatric populations, foster youth, and other underrepresented populations, telehealth may become a common treatment delivery option. Future research should evaluate what participant characteristics are necessary for telehealth (e.g., low levels of challenging behavior, ability to attend to relevant stimuli, compliance with one-step instructions) and what types of interventions are feasible to implement remotely. Future research should also evaluate caregiver prerequisites. The caregivers we trained were able to implement various interventions using vocal instructions, textual instructions, or both, but some caregivers may need additional supports. As research in this area expands, practitioners will have better resources to guide their implementation of various behavior analytic procedures.

In addition to conducting FAs and implementing communication training remotely, future research should explore a wider variety of interventions such as teaching interview skills (Edgemon et al., 2020) , chaining procedures, social skills training, self-management, and tolerance training, among others. Some recent studies (Fisher et al., 2020; Pellegrino & DiGennaro-Reed, 2020; Piazza et al., 2021) have begun to demonstrate the effectiveness of telehealth interventions, but the research base is still limited. As researchers continue to evaluate interventions implemented remotely, practitioners will have a clearer guide on the most effective strategies. Although descriptive in nature, this review could also serve as a foundation for systematic investigation of the various assessments and interventions utilized in behavior analytic telehealth practice as well as participants for whom telehealth could be beneficial.

BACB certificant data

Combining stimulants with antipsychotics for foster children and adolescents: Do prescribers adhere to guidelines? Children & Youth Services Review, 121. Advance online publication

Application of multiple baseline designs in behavior analytic research: Evidence for the influence of new guidelines

Practice parameters for telehealth-implementation of applied behavior analysis: Continuity of care during the COVID-19 pandemic

Behavioral skills training to increase interview skills of adolescent males in a juvenile residential treatment facility

Telehealth as a model for providing behavior analytic interventions to individuals with autism spectrum disorder: A systematic review

Visual aids and structured criteria for improving visual inspection and interpretation of single-case designs

A randomized clinical trial of a virtual-training program for teaching applied-behavior-analysis skills to parents of children with autism spectrum disorder

A statewide child telepsychiatry consult system yields desired health system changes and savings. Telemedicine & e-Health

Impact of COVID-19 on the applied behavior analysis workforce: Comparison across remote and nonremote workers. Behavior Analysis in Practice

Single-case designs technical documentation. What Works Clearinghouse

Using the dualcriteria methods to supplement visual inspection: An analysis of nonsimulated data

Foster children's behavioral disturbance and detachment from caregivers and community institutions

Behavior analytic contributions to public health and telehealth

Guidelines for developing telehealth services and troubleshooting problems with telehealth technology when coaching parents to conduct functional analyses and functional communication training in their homes

Remote coaching of caregivers via telehealth: Challenges and potential solutions

Telehealth and autism: Treating challenging behavior at lower cost

Alabama psychiatric medication review team (APMRT): Advocating for foster children

Enhancing the lives of foster youth with behavioral interventions

Variations in mental health diagnosis and prescribing across pediatric primary care practices

Fidelity outcomes for autism-focused interventionists coached via telepractice: A systematic literature review

Using telehealth to teach valued skills to adults with intellectual and developmental disabilities

Effective remote staff training of objective session notes

Ethical considerations in the design and implementation of a telehealth service delivery model

Best practices and considerations for effective service provision via remote technology

Ethical considerations when delivering behavior analytic services for problem behavior via telehealth

Preliminary evidence on the use of telehealth in an outpatient behavior clinic

Training individuals to implement applied behavior analytic procedures via telehealth: A systematic review of the literature

Effectiveness and acceptability of parent training via telehealth among families around the world

Caregiver training via telehealth on behavioral procedures: A systematic review

Evaluation of a foster parent intervention for foster children with externalizing problem behavior

Conducting functional communication training via telehealth to reduce the problem behavior of young children with autism

Conducting functional analyses of problem behavior via telehealth

Psychotropic medication patterns among youth in foster care

Psychotropic polypharmacy in the US pediatric population: A methodological critique and commentary

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations