key: cord-0730043-ujru43fr
authors: McKee, Grace B.; Pierce, Bradford S.; Donovan, Emily K.; Perrin, Paul B.
title: Examining models of psychologists' telepsychology use during the COVID‐19 pandemic: A national cross‐sectional study
date: 2021-05-24
journal: J Clin Psychol
DOI: 10.1002/jclp.23173
sha: 8870f2e82d4c959f455dc6d250afd0e9052b9fba
doc_id: 730043
cord_uid: ujru43fr

OBJECTIVE: The aim of this study is to apply the theory of reasoned action (TRA) and technology acceptance model (TAM) to psychologists' telepsychology use during the COVID‐19 pandemic. METHODS: A sample of 2619 US‐licensed psychologists completed a survey assessing telepsychology use and aspects of both models in May 2020. RESULTS: Cross‐sectional TRA and TAM path models evidenced excellent fit in explaining psychologists' telepsychology use. The TRA indicated that psychologists' attitudes concerning telepsychology and subjective norms were associated with intentions to use telepsychology, which related to percentage of clinical work performed via telepsychology. The TAM showed that perceived usefulness of telepsychology and perceived ease of use were associated with attitudes toward telepsychology. Perceived usefulness was associated with psychologists' intention to use telepsychology, as was perceived ease of use. CONCLUSION: Efforts to facilitate telepsychology provision during the pandemic and broadly may benefit from trainings and campaigns to address attitudes toward telepsychology, subjective norms, and perceived ease of use.

training, handling client safety/crisis, insufficient demand, efficacy) and environmental/policy concerns (e.g., privacy, legality, ethics, reimbursement, prohibitive organizational policies/culture; Pierce et al., 2020c) .

Moreover, several theoretical models of behavior change may provide a useful framework for explaining the adoption of telepsychology among psychologists. The theory of reasoned action (TRA) outlines a framework for understanding the relationships among attitudes toward behavior, perceived social pressure (i.e., subjective norms), behavioral intentions, and behavior (Fishbein & Ajzen, 1977) . The TRA posits that attitudes and norms precede intentions, which precede behavior (Fishbein & Ajzen, 1977) . Derived from the TRA, the technology acceptance model (TAM) was developed to provide a foundation for understanding factors associated with an individual's willingness to adopt computer technologies or why technology has not been accepted in a given population (Davis et al., 1989) . Perceived usefulness (i.e., expectation of technology increasing/decreasing performance) and perceived ease of use (i.e., expectation of effort level required and self-efficacy surrounding use of the technology) are key variables related to computer technology acceptance (Davis et al., 1989) . Both theories posit that attitude toward technology influences behavioral intention to use it, but the TAM does not measure subjective norms and adds perceived usefulness as an influence on behavioral intention. In other words, the theories diverge in important ways with regard to which variables might kick off the chain of events leading to attitudes, intention, and actual behavior.

The TRA and TAM previously have been used to examine psychologists' openness to using telepsychology before the COVID-19 pandemic (Pierce et al., 2020b) . Using data from 1791 psychologists currently practicing psychotherapy, Pierce et al. (2020b) developed and analyzed path models derived from the TRA and TAM to assess hypothesized associations among attitudes toward telepsychology, subjective norms, intention to use telepsychology, perceived ease of use, perceived usefulness, and current use of telepsychology. Both the TRA and the TAM models poorly fit the data, and the authors hypothesized that this could be due, in part, to the dichotomous (yes/no) outcome used for use of telepsychology. A third model was developed, which combined the two theoretical models and resulted in adequate to good fit. Within this combined model, variables associated with use of telepsychology by psychologists were perceived attitudes/expectations of others (i.e., subjective norms) that were associated with psychologists' perceptions of telepsychology's usefulness and ease of use. Furthermore, as this study was conducted before the COVID-19 pandemic, only 20% of participants reported telepsychology use. It is unclear whether these associations would be the same during the COVID-19 "telemedicine revolution" , when many psychologists transitioned some or all of their clinical services to telepsychology. Moreover, given that mental health providers tend to report more positive attitudes toward telepsychology after use (Brooks et al., 2012) and that perceived ease of use also appears to increase (Gibson et al., 2011) , it may be that associations among TAM and TRA variables may differ from these previous patterns in the first few months of the pandemic, when telepsychology was being widely implemented.

There is potential for telepsychology to aid psychologists in increasing the reach of mental health services during the COVID-19 pandemic and beyond, given the profound rates of mental health issues identified on a global scale (Alzueta et al., 2021) . Furthermore, telepsychology allows psychologists to maintain the treatment of their clients while following social distancing recommendations. Although telepsychology delivered via telephone or video conferencing technology has been proven effective and non-inferior to in-person therapy for a wide range of disorders (Varker et al., 2019) , many psychologists did not offer these services before the COVID-19 pandemic.

The purpose of the present study was to test the applicability of the TRA, TAM, and previously combined model (Pierce et al., 2020b) to psychologists' adoption of telepsychology in the context of the COVID-19 pandemic. While the TAM, TRA, and combined model have been used to examine the uptake of new technology in the context of telehealth broadly and telepsychology specifically (Monthuy-Blanc et al., 2013; Pierce et al., 2020b) , the COVID-19 pandemic may have profoundly impacted the landscape of subjective norms and attitudes contributing to behavior change, thus necessitating a new examination at this unique point in the history of mental healthcare delivery.

1 | METHODS

The data used in this study are a secondary analysis of participant responses obtained from a larger study of environmental and demographic factors relating to psychologists' use of telepsychology during the COVID-19 pandemic (Pierce, Perrin, Tyler, et al., 2020) . A recruitment list of names and email addresses of potential participants was generated using publicly available information on websites for mental health clinics and counseling centers, professional listservs, and professional organization directories. Criteria for eligibility were that participants must be licensed to practice as a psychologist in the United States at the time of participation, age 18 or older, and actively practicing as a psychologist in the United States at the time of participation. A recruitment flow diagram is shown in Figure 1 . Emailed invitations were sent to 27,324 potential participants between May 11, 2020, and May 25, 2020. Among that number, 14.3% (n = 3897) of invitations returned as undeliverable, whereas 15.8% (n = 3694) of delivered emails resulted in individuals at least accessing the study's online information sheet.

Among these, 184 individuals chose not to consent to participate in the survey or abandoned the survey after reviewing the information sheet. Another 541 began the survey but failed to complete it. In addition, 198 failed to meet eligibility criteria based on their responses to the initial screener items. Among the remainder, 127 were excluded from analysis after refusing to provide demographic data (e.g., age, years in practice). Finally, data were excluded for 25 participants who indicated they had started practicing as a licensed psychologist before age 18 or failed to provide a meaningful answer to the final validity check question, "In one sentence or less, what do you

think the purpose of this study was?" This resulted in an 11.18% (n = 2619) response rate of eligible individuals who fully participated in the study. This aligns with research examining response rates using similar recruitment strategies (Petrovčič et al., 2016) .

Women comprised 64.2% of the sample and there was a mean age of 57.29 years, with an average of 24.22 years in practice. Full sample demographics appear in Table 1 . These sample characteristics were similar to APA's 2017 membership demographics (American Psychological Association, 2018), although the geographical region was not surveyed and therefore could not be assessed for representativeness. In addition, respondents were presented with 20 theoretical therapeutic approaches and asked to select all approaches that applied to them; cognitive behavioral therapy (70.4%), psychodynamic (38.2%), interpersonal (31.1%), family systems (23.9%), and solution-focused (23.7%) were the five most commonly identified therapeutic approaches, although all 20 were endorsed by at least some of the sample.

The current study was approved by the host University's Institutional Review Board. Personalized email invitations were sent to potential participants, requesting them to complete the survey. The survey's purpose was explained as an attempt to understand approaches to treating their patients during the COVID-19 pandemic, but telepsychology was not mentioned to avoid introducing any bias or assumptions about the study or willingness to participate. To improve participation rates, a reminder invitation was sent one week after the initial invitation. Invited individuals who clicked on the link to the Qualtrics platform were provided an information sheet with further details about the study, including IRB information and contact details for the investigators, before asking them to provide informed consent to participate.

Participants completed a number of demographic questions about themselves and their practice, read the telepsychology definition used in this survey, and then answered the items concerning telepsychology. Participants were given the following description of telepsychology, "For the purpose of this survey, 'telepsychology' refers the use of realtime audio (e.g., telephone) and/or video conferencing technology to provide psychological services." This description aligns with definitions of telehealth provided by both the Centers for Medicare and Medicaid Services (2019) and the U.S. Drug Enforcement Administration Diversion Control Division (2019). For the current study, participants provided answers based on their experiences "during the COVID-19 pandemic in the United States."

Participants provided data regarding their age, gender, race/ethnicity, years of clinical practice, current primary practice setting, and percentage of clinical work conducted via telepsychology. The level of telepsychology use was assessed by asking participants, "What percentage of your patient treatment is provided using telepsychology?" with answers ranging from 0% to 100%.

Data were collected using an item pool originally developed by Chau and Hu (2002) to assess variables within both the TAM and TRA to understand healthcare professionals' use of technology to deliver health care. These items have demonstrated good validity with healthcare professionals when measuring attitudes, subjective norms, perceived usefulness, perceived ease of use, and behavioral intention (Chau & Hu, 2002) . To prevent participant attrition, five items with the highest factor loading for each construct (Chau & Hu, 2002) were selected rather than | 7 using the entire 18-item pool. In these items, the words "telemedicine technology" were replaced with "telepsychology." For example, the item assessing perceived usefulness read "Using telepsychology can improve my patient care and management." Participants answered TAM and TRA items using 7-point Likert-type scales, where possible answers ranged from 1 "strongly agree," to 4 "neutral," to 7 "strongly disagree."

Cross-sectional path models were estimated in IBM AMOS 24.0 using maximum likelihood estimation. Initially, fully saturated versions of each model were run for both the TRA and the TAM by including all theorized variables and all possible paths. Following procedures discussed by Meyers et al. (2017) , these saturated models were trimmed by removing the path that would least significantly decrease R 2 and then re-estimated. Models were trimmed until the only paths that remained were statistically significant and had standardized regression weights of 0.1 or greater, representing at least a small effect size. As an exploratory analysis, we also estimated the crosssectional path model combining the TAM and TRA that was previously found to be a better fit than either the TAM or the TRA (Pierce et al., 2020b) .

Once final models were determined, they were compared using a number of goodness-of-fit criteria including fit cutoffs of 0.90 for the normed fit index (NFI) and incremental fit index (IFI; Bentler & Bonett, 1980) , 0.95 for the Tucker-Lewis Index (TLI) and comparative fit index (CFI; Hu & Bentler, 1999) , and 0.06 for the root mean square error of approximation (RMSEA; Hu & Bentler, 1999 ). Table 2 shows the descriptive statistics and bivariate relationships among the study variables. Attitude toward use of telepsychology, subjective norms, perceived usefulness, perceived ease of use, behavioral intention to use telepsychology, and current percentage use of telepsychology were all significantly positively associated. Table 3 shows mean scores of study variables grouped according to reported primary practice setting and number of years in practice. As reported previously (Pierce, Perrin, Tyler, et al., 2020) , psychologists reported performing 7.07%

(SD = 14.86) of their clinical work using telepsychology before the pandemic, with 45.70% not using telepsychology at all during that time. Psychologists in Veterans Affairs medical centers and individual practices reported the highest percentage use of telepsychology before the pandemic. Subjective norms were associated with intention to use telepsychology, β = 0.24, p < 0.001, and with current use, β = 0.12, p < 0.001. Attitude toward telepsychology was associated with intention to use telepsychology, β = 0.50, p < 0.001, but not with current use, β = −0.002, p = 0.937. Intention to use telepsychology was associated with current use, β = 0.53, p < 0.001. Fit indices were invalid because the model was saturated.

The (Table 4 ). Subjective norms were again associated with behavioral intention to use telepsychology, β = 0.24, p < 0.001, and with current use, β = 0.12, p < 0.001. Attitude toward telepsychology was also associated with intention to use telepsychology, β = 0.50, p < 0.001, which was, in turn, associated with current use, β = 0.53, p < 0.001. In addition, there were significant indirect effects (all ps < 0.001) of subjective norms on current use, β = 0.13, and of attitude on current use, β = 0.26, both through intention to use telepsychology.

The fully saturated, cross-sectional TAM model explained 33.1% of the variance in percentage use of telepsychology. Within this model, perceived usefulness of telepsychology was associated with attitude toward telepsychology, β = 0.47, p < 0.001, and with intention to use telepsychology, β = 0.39, p < 0.001, but not with percentage use, β = 0.04, p = 0.417. Perceived ease of use was also associated with attitude toward telepsychology, β = 0.15, p < 0.001, intention to use telepsychology, β = 0.11, p < 0.001, and percentage use, β = 0.04, p = 0.036.

Attitude toward telepsychology was associated with intention to use telepsychology, β = 0.35, p < 0.001, which was, in turn, associated with percentage use, β = 0.55, p < 0.001. Again, fit indices were invalid because the model was 

The first trimmed TAM model was constructed by removing the path from the saturated model that would least decrease R 2 , which was the effect of perceived usefulness of telepsychology on current use. This model still explained 33.1% of the variance in percentage use of telepsychology and evidenced excellent fit (Table 4) . Although all paths were statistically significant at p < 0.05, the effect of ease of use on current use was below the predetermined standardized regression weight magnitude, β = 0.04, p = 0.024, and was subsequently removed for the final model.

The final model ( Figure 3 ) explained 33.0% of the variance in percentage use of telepsychology and also evidenced excellent fit. Perceived usefulness of telepsychology was associated with attitude toward telepsychology, β = 0.47, p < 0.001, and intention to use telepsychology, β = 0.39, p < 0.001. Ease of use was associated with attitude toward telepsychology, β = 0.15, p < 0.001, and intention to use telepsychology, β = 0.11, p < 0.001. Attitude toward telepsychology was associated with intention to use telepsychology, β = 0.35, p < 0.001, which was, in turn, associated with percentage use, β = 0.57, p < 0.001. There were also a number of significant indirect effects (all ps < 0.001). Ease of use exerted an indirect effect on intention through attitude, β = 0.05, and current use through both attitude and intention, β = 0.10. Perceived usefulness exerted an indirect effect on intention through attitude, β = 0.16, and on current use 

The cross-sectional combined model (Figure 4) 

In examining the fit of the final versions of the TRA and TAM models and Model 2 of the TAM (Table 4 ), the TRA model demonstrated slightly better fit to the data, as shown by the NFI, IFI, TLI, CFI, and RMSEA indices. However, both final models showed excellent fit across all indices; as neither showed substantially better fit than the other, both were retained, suggesting that both models are useful in explaining use of telepsychology during the pandemic. In contrast, the fit indices for the combined model ranged from poor to adequate fit, which may suggest that it does not describe telepsychology use as well as the TRA or TAM models.

The current cross-sectional study is the first known to examine the uptake of telepsychology services among psychologists during the pandemic using theoretical models of behavior change. The TRA and the TAM both demonstrated excellent fit in explaining psychologists' percentage use of telepsychology during the pandemic. All possible indirect effects, mediational and multiple mediational, within each model were statistically significant.

Results from the TRA indicated that psychologists' attitudes concerning telepsychology and subjective norms were both significantly associated with their intentions to use telepsychology. Of these two factors, attitudes showed the strongest association with intention to use telepsychology, and the association between attitude toward telepsychology and current use was fully mediated by intention to use telepsychology. Subjective norms F I G U R E 4 Combined model [Color figure can be viewed at wileyonlinelibrary.com]

were associated with use of telepsychology, both directly and indirectly through their statistical effect on intention to use telepsychology, reflecting a partial mediation. In turn, intention to use telepsychology was associated with actual percentage use of telepsychology during the pandemic, explaining 34% of the variance in current use. The exploratory trimming procedure used in the current study also demonstrated that subjective norms were directly related to percentage use of telepsychology, an association that is not directly theorized in the TRA.

Several of these associations were also found by Pierce et al. (2020b) in a pre-pandemic path analysis of psychologists' openness to use telepsychology, although the current model explained more than twice the variance explained by the TRA in that previous study. As in the current study, psychologists' intention to use telepsychology was significantly associated with attitudes toward telepsychology and, to a much lesser degree, by perceived subjective norms; however, in the current study, intention to use telepsychology was less strongly associated with attitude toward telepsychology (β = 0.50 vs. β = 0.67, respectively). Intention to use was also more strongly associated with subjective norms (β = 0.24 vs. β = 0.06, respectively) than in the previous study. In turn, intention to use was associated with actual use, and to a greater degree than in the previous study (β = 0.53 vs. β = 0.41, respectively). However, that model did not include the direct association between subjective norms and current use found in the current model, as it was not theorized in the original TRA, which may partly explain why the model's fit indices ranged from good to poor. Moreover, actual use of telepsychology was dichotomized (yes/no), in contrast to the current study, which used a continuous measure of current use.

Results from the TAM showed that psychologists' perceived usefulness of telepsychology and perceived ease of use both were associated with their attitudes toward telepsychology, although perceived usefulness demonstrated a much stronger statistical effect. Perceived usefulness was also related to psychologists' reported intention to use telepsychology, an association that was posited in the original version of the TAM (Davis, 1986) but was removed in an early revision (Davis et al., 1989) . This particular finding is consistent with the author's initial supposition that within an organizational setting, individuals will intend to take up technology that they believe will improve their job performance, and that this belief is distinct from positive or negative attitudes toward technology more generally.

Interestingly, the exploratory trimming procedure revealed perceived ease of use as significantly associated with intention to use telepsychology, even though it was not directly theorized in the TAM. Although it was not as strongly related to intention as was perceived usefulness or attitude toward telepsychology, it did represent a unique effect that was not fully explained by attitude toward telepsychology. Again, perceived usefulness showed a stronger association with intention to use telepsychology than did perceived ease of use, although both were significant. Both of these associations, even though not posited in earlier versions of the TAM, were suggested as potential revisions in a more recent meta-analysis of associations among the TAM variables in studies of technology uptake (Schepers & Wetzels, 2007) ; that study also found that perceived usefulness showed a stronger association with behavioral intention than ease of use. Moreover, all mediational and multiple mediational indirect effects within the model were significant, providing internal evidence supporting the overall structure of the model. Taken together, these findings suggest that psychologists' beliefs that telepsychology would positively impact their work and that it would be reasonably easy to implement each impact their attitudes about telepsychology as well as their behavioral intention to use it during the pandemic.

The results from the TAM showed similarities to some pre-pandemic studies on telepsychology and telehealth uptake among providers. For example, Pierce et al. (2020b;  path analytic) also found that psychologists' attitudes toward telepsychology were strongly informed by their perceptions of usefulness and, to a lesser degree, by perceived ease of use. Psychologists' attitudes toward telepsychology and perceived usefulness also showed direct associations with intention to use, which, in turn, was associated with actual use. Again, the model in the current study explained nearly two times as much variance in current use as did the model in the previous study. Associations in the current study between attitudes toward telepsychology and perceived usefulness (β = 0.47 vs. β = 0.66, respectively) and attitudes toward telepsychology and perceived ease of use (β = 0.15 vs. β = 0.21, respectively) were weaker than those found in the previous study. Likewise, the association between attitude toward telepsychology and intention to use was also somewhat weaker than that found previously (β = 0.35 vs. β = 0.44,

respectively), although, as with the TRA, the association found in the current study between intention to use and actual use was stronger than that found previously (β = 0.57 vs. β = 0.41, respectively). However, the previous model demonstrated adequate to poor fit to the data, possibly due to omitting significant paths that were not originally theorized in the model, such as the direct path from perceived ease of use to intention to use telepsychology that was significant in the current model. Finally, several of the path coefficients in the current TAM were similar in magnitude to those found in a meta-analysis of TAM research (Schepers & Wetzels, 2007) , in particular the strong association between behavioral intention to use and actual use (β = 0.57 in the current study vs. β = 0.55 in the meta-analysis). Taking into account results from the previous meta-analysis and the current study, it may be that the TAM could be expanded to account for these direct associations between behavioral intention to use and perceived usefulness and ease of use, respectively.

In contrast, the current model differed in several ways from that found by Monthuy-Blanc et al. (2013) in a study of mental health providers' attitudes toward telepsychology in the treatment of First Nations patients. In that study, the primary predictor of intention to use telepsychology was the providers' perceptions of usefulness, rather than attitudes toward telepsychology or perceived ease of use. In fact, attitude toward telepsychology did not significantly predict intention to use at all. Moreover, in a previous study of Hong Kong physicians' receptivity to telehealth, perceived usefulness significantly predicted both attitude and intention but perceived ease of use did not .

Overall, the findings from both models demonstrate a strong connection between intention to use and actual current use of telepsychology. Attitudes toward telepsychology were highly associated with intention to use telepsychology in both models, suggesting that psychologists' beliefs about telepsychology (e.g., endorsement of myths or stereotypes about telepsychology provision, such as beliefs about inadequacy of crisis intervention options or that serious mental disorders cannot be adequately treated via telepsychology; Fenichel et al., 2002) substantially inform their intention to use it in clinical care. Perceived usefulness and subjective norms were also associated with intention to use telepsychology, as was perceived ease of use, though to a lesser degree.

In contrast, the combined model tested previously (Pierce, Perrin, Tyler, et al., 2020) evidenced less than adequate fit to the data in the current study. As discussed above, these differences may be due, in part, to the previous dichotomization of the outcome variable of current use of telepsychology. However, it may also indicate that the variables included in the path model do not adequately describe use of telepsychology during the pandemic. Absent from this model is intention to use telepsychology, which was strongly associated with actual use of telepsychology, both in the models presented in the current study as well as in previous research (e.g., Schepers & Wetzels, 2007) . It could be that psychologists' intention to use telepsychology was a more salient factor associated with actual use due to conditions during the pandemic, in comparison to before the pandemic, in which intention to use telepsychology may not have been sufficient to predict actual use when subjective norms, perceived ease of use, and perceived usefulness were taken into account. For example, before the pandemic, in the absence of systemic and widespread need for alternatives to face-to-face clinical care, these other factors may have been more relevant than intention for actual use.

Additional research will be necessary to examine these associations and the applicability of both models for psychologists' use of telepsychology once the pandemic has subsided and it is generally safe to provide in-person services. It could be, for example, that perceived usefulness and ease of use of telepsychology may become even more salient to psychologists' decisions to provide care via telepsychology when alternative methods of care delivery are more readily available. Alternatively, it could be that usefulness and ease of use could demonstrate greater associations with telepsychology use during the pandemic; for example, psychologists who endorse negative beliefs about telepsychology usefulness or ease of use may have chosen to scale down their clinical practice during the pandemic rather than increase use of telepsychology. Earlier pre-pandemic research has suggested that providers' attitudes toward telepsychology tend to improve over time with use (e.g., Brooks et al., 2012) , which may relate to their increased comfort level with using synchronous video technology (Gibson et al., 2011) .

However, it is unclear whether those patterns will continue post-pandemic once providers and patients have the option to safely return to in-person services.

Taken together, these findings suggest several important avenues for increasing or improving use of telepsychology among psychologists during the COVID-19 pandemic and more broadly. First, efforts to increase telepsychology provision may benefit from training or campaigns to address attitudes toward telepsychology, such as by debunking myths about telepsychology or challenging biases against it. Second, training could target subjective norms by increasing providers'

awareness of the prevalence of telepsychology; similarly, organizations with established telepsychology infrastructure could promote awareness of provider resources, such as training in telepsychology and availability of supportive policies and procedures, or even incentivize providers' use of telepsychology for appropriate clinical cases. There are a number of resources available through the APA website (American Psychological Association, 2020) that could help address these first two issues, including several articles and internal APA surveys of telepsychology provision during the pandemic.

Many of these resources serve to underscore the ubiquity of telepsychology during COVID-19 and either directly or indirectly address many common misperceptions of telepsychology. Other organizations (e.g., VHA, large hospital systems) also have trainings available for providers interested in learning more. Given that psychologists across different practice settings tended to report similar mean scores on the study variables, it may be that similar strategies could be used across settings; however, psychologists' scores appeared to differ more for those with more than 25 years of clinical experience, which could suggest that different strategies may be needed to target less versus more experienced psychologists. Empirical research would be needed to confirm whether either of these patterns is associated with differential effectiveness of interventions to increase telepsychology use.

These findings also convey a number of important implications for future telepsychology research and policy.

There may be several avenues for policymakers to support the continued use of telepsychology at high rates postpandemic. For example, maintaining changes made by the U.S. Centers for Medicare and Medicaid Services (2020) to continue to reimburse telepsychology at the same rate as in-person visits has the potential to influence providers' perceptions of the usefulness of telepsychology and perhaps their attitudes toward telepsychology more generally. Making permanent some of the temporary changes to the Health Insurance Portability and Accountability Act and to state board requirements for licensure for provision of services across state lines has the potential to ameliorate concerns related to legality and confidentiality reported by mental health providers in a systematic review of attitudes toward telepsychology (Connolly et al., 2020) .

As these implications are based on cross-sectional analyses, we caution that future research should test the effectiveness of these potential interventions or policy changes using experimental or quasi-experimental methodology that would enable researchers to estimate their causal impact. This reflects an important direction for additional pandemic-related research but also will be a crucial target for investigation once safety concerns related to the pandemic have been ameliorated. It could be, for example, that these interventions may have more or less impact on telepsychology uptake post-pandemic when providers and patients have more choices about modality of care. Effects on uptake may also depend in part on whether post-pandemic healthcare policy continues to support the widespread use of telepsychology. Second, although the initial variables included in each model were based on the TRA and TAM theoretical models, respectively, the final models were derived using exploratory trimming procedures and do not exactly replicate the theoretical associations among variables as posited by the TRA and TAM. So the trimming procedure could have resulted in a better fit than if a strict adherence to all paths, or lack thereof, in the models had occurred (e.g., Pierce et al., 2020b) . Finally, as the data in the current study were cross-sectional and the path analyses were correlational in nature, causality cannot be proven, and it cannot be assumed that changes in use of telepsychology were directly caused by the other variables. Alternative theoretical models may be operating among the variables. For example, it may be the case that attitudes and beliefs about telepsychology were impacted as psychologists were forced to increase use of telepsychology to comply with social distancing guidelines. Further, although the mediational effects found in the current study were statistically significant, this does not necessarily suggest that these cross-sectional associations also operate in a temporal or casual fashion. Cross-sectional mediational analyses have the potential to generate biased estimates (e.g., Maxwell & Cole, 2007) , and so it would be important to replicate this study using longitudinal data.

Further, future research employing longitudinal designs could use cross-lagged panel approaches to more closely examine questions of causality as well as mediation.

Although the current study used a definition of telepsychology that was consistent with CMS definitions ("real-time audio [e.g., telephone] and/or video conferencing technology"), the survey did not assess for psychologists' provision of clinical care via other modalities, such as interactions with a patient via online chat, email, or text message, either as a standalone treatment or as an adjunct to more traditional care; previous research suggests that a sizeable minority of psychologists may use these asynchronous modalities to provide care (e.g., Glueckauf et al., 2018) . Moreover, patients may also seek care through mental health apps and webbased interventions. While research is already under way to explore factors related to healthcare providers' uptake of these m-health services (see Gagnon et al., 2016 , for a review), it would be important to examine whether the results from the current study replicate within a similar sample of US-licensed psychologists providing care during the pandemic. It could be that demographic factors (such as patient or provider rurality, age, income level, employment status, health insurance status) and TRA or TAM variables may be associated with provision of care through these channels. Furthermore, although testing for differences in study variables based on respondent characteristics was beyond the scope of the current study, the parent study found some differences in telepsychology uptake based on primary practice setting (Pierce, Perrin, Tyler, et al., 2020) , and so additional studies may benefit from examining how these theoretical variables may differ as a function of demographic or practice-related characteristics.

Despite these limitations, the current study is the only study to date examining psychologists' provision of telepsychology services during the COVID-19 pandemic using theoretical models of behavior change. Furthermore, it is the largest known survey of licensed psychologists practicing in the United States during the pandemic. The excellent fit of both the TRA and TAM path models suggests extremely ripe targets for training and campaigns to increase and improve psychologists' provision of telepsychology during the pandemic and beyond.

Priorities for the US health community responding to COVID-19

How the COVID-19 pandemic has changed our lives: A study of psychological correlates across 59 countries

Guidelines for the practice of telepsychology

Demographic characteristics of APA members by membership status

Psychological services via telehealth: Information for consumers

Significance tests and goodness-of-fit in the analysis of covariance structures

The diffusion of telehealth in rural American Indian communities: A retrospective survey of key stakeholders

Resources for clinics and healthcare facilities

Investigating healthcare professionals' decisions to accept telemedicine technology: An empirical test of competing theories

A systematic review of providers' attitudes toward telemental health via videoconferencing

A technology acceptance model for empirically testing new end-user information systems: theory and results (Doctoral dissertation)

User acceptance of computer technology: A comparison of two theoretical models

EFPA psychologists' support hub about COVID-19 crisis

Myths and realities of online clinical work

Belief, attitude, intention, and behavior: An introduction to theory and research

m-Health adoption by healthcare professionals: A systematic review

Mental health professionals' perspectives of telemental health with remote and rural First Nations communities

Survey of psychologists' telebehavioral health practices: Technology use, ethical issues, and training needs

Outcomes of 98,609 U.S. Department of Veterans Affairs patients enrolled in telemental health services

Leveraging parent-child interaction therapy and telehealth capacities to address the unique needs of young children during the COVID-19 public health crisis

Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives

Examining the technology acceptance model using physician acceptance of telemedicine technology

Joint Task Force for the Development of Telepsychology Guidelines for Psychologists

Telepsychology practice: Primer and first steps

Bias in cross-sectional analyses of longitudinal mediation

Applied multivariate research: Design and interpretation (Version 3rd) [Computer software

Factors influencing mental health providers' intention to use telepsychotherapy in First Nations communities

fight against coronavirus, telemedicine ramps up at Johns Hopkins

Tracking COVID-19's impact on global internet performance

COVID-19 and telemedicine: A revolution in healthcare delivery is at hand

Rapid telepsychology deployment during the COVID-19 pandemic: A special issue commentary and lessons from primary care psychology training

The effect of email invitation elements on response rate in a web survey within an online community

Demographic, organizational, and clinical practice predictors of U.S. psychologists' use of telepsychology

Path analytic modeling of psychologists' openness to performing clinical work with telepsychology: A national study

Pre-COVID-19 deterrents to practicing with videoconferencing telepsychology among psychologists who didn't

The COVID-19 telepsychology revolution: A national study of pandemic-based changes in U.S. mental health care delivery

A meta-analysis of the technology acceptance model: Investigating subjective norm and moderation effects

Summary of board actions-COVID-19 State of emergency license application waivers/temporary licensure/provisional licensure

How video appointments are changing the way Mayo Clinic patients receive care

VA push for telehealth amid coronavirus slowed by balky networks

Emergency licensure for practice within a declared disaster area

Medicare telemedicine health care provider fact sheet

OCR eases HIPAA telehealth enforcement for COVID-19 emergency

Title 21 United States Code (USC) Controlled Substances Act

Suddenly becoming a "virtual doctor": Experiences of psychiatrists transitioning to telemedicine during the COVID-19 pandemic

Efficacy of synchronous telepsychology interventions for people with anxiety, depression, posttraumatic stress disorder, and adjustment disorder: A rapid evidence assessment

Veterans Health Administration, Office of Emergency Management

WHO Director-General's opening remarks at the media briefing on COVID-19 -11