key: cord-0882414-yv1hs049
authors: Fung, Alex; Irvine, Mike; Ayub, Aysha; Ziabakhsh, Shabnam; Amed, Shazhan; Hursh, Brenden E.
title: Evaluation of telephone and virtual visits for routine pediatric diabetes care during the COVID-19 pandemic
date: 2020-10-12
journal: J Clin Transl Endocrinol
DOI: 10.1016/j.jcte.2020.100238
sha: f7f08686ebba311874b548b7f3b068118077bb37
doc_id: 882414
cord_uid: yv1hs049

AIMS: To evaluate pediatric type 1 diabetes telehealth visits during the COVID-19 pandemic, with a focus on assessing the usability of these visits and gathering patient perspectives. METHODS: An online survey, which included a validated telehealth usability questionnaire, was offered via email to families with a telephone or virtual visit since the COVID-19-related cancellation of routine in-person care. Survey data was linked with the British Columbia (BC) Clinical Diabetes Registry. Outcomes between groups were assessed using Welch’s t-test. Associations with type of visit as well as with desire to return to in-person care were assessed with logistic regression models. RESULTS: The response rate was 47%. Of 141 survey respondents, 87 had clinical data available in the BC Clinical Diabetes Registry, and thus were included in our analysis. Overall, telephone and virtual visits were rated highly for usability. Telephone visits were easier to learn to use, and simpler to understand; however, telephone and virtual visits were similar across multiple areas. No factors associated with choosing one type of visit over the other, or with desire to return to in-person care, could be identified. 72% of participants want future telehealth care; however, some would like all future care to be in-person. CONCLUSIONS: Telephone and virtual visits had impressive usability. Many families want telehealth to play a significant part in their future care.

In British Columbia (BC), more than 2000 children and adolescents have type 1 diabetes mellitus (T1D), a serious disease that requires intensive daily medical management. Pediatric diabetes has remained primarily an 'in-person' specialty across Canada, and in line with that, the diabetes team at BC Children's Hospital (BCCH) sees more than 1400 in-person pediatric diabetes visits per year. Up until now, only a small proportion of patients at BCCH were seen by virtual care, and this service was limited to patients living far away from our hospital. These visits accounted for approximately 5% of all visits, and only 1 out of 8 physicians in the division participated in virtual health prior to the novel coronavirus (COVID- 19) pandemic. During the early stages of the COVID-19 pandemic, and in line with provincial health mandates, the diabetes team rapidly shifted to performing 100% of routine diabetes clinic visits by either telephone or virtual technology.

Recent work has revealed that healthcare professionals have been observing an increased incidence of diabetic ketoacidosis in youth [1] [2] [3] , possibly attributed to parents avoiding hospital visits due to the concern of potential infection with COVID-19 [3] . Similarly, for individuals living with diabetes, the COVID-19 pandemic is correlated with increased variability in glycemic control and a reduction in physical activity [4] , which may be a result of stay-at-home recommendations. However, the International Society of Pediatric and Adolescent Diabetes (ISPAD) has highlighted both a critical need for continued diligence in providing standard pediatric diabetes care to avoid health complications and hospitalizations, as well as the importance of ongoing collaboration between patients, families and healthcare providers [5] . ISPAD has reported a swift increase in utilization of telehealth as a strategy for healthcare professionals to continue to provide care for children and adolescents with diabetes [5] , and this transition from in-person visits to virtual health technologies during COVID-19 seems to be a global trend [1] .

Despite the recent rapid uptake of virtual care in the setting of the COVID-19 pandemic, there are no long-term studies describing virtual health services for pediatric diabetes [6] . The use of virtual care in young adults aged 18-25 has demonstrated that telehealth keeps patients engaged in their diabetes care, increases treatment adherence and is overall feasible and acceptable in this population [6, 7] . In the pediatric age range, it has been reported that telehealth increases access to healthcare and may improve overall diabetes care [8, 9] . Of specific interest to our study team in the current context of the rapid shift to virtual care during the pandemic, is the concept of usability. Usability refers to the extent to which a product is used "to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context" [10] . Usability has been identified as an enabler of successful telehealth care, and may reduce burden on patients and clinicians [11] ; however, there remains limited information about the usability of virtual technologies for pediatric patients and their families, especially during a pandemic.

As there has been an unprecedented and rapid shift to a new model of diabetes care delivery, we set out to understand these visits from the patient's perspective. The objectives of this study were to evaluate the usability of diabetes telephone and virtual visits and to identify any differences in usability between these two modes of care. We also set out to describe patient characteristics that might predict a preference for virtual versus telephone visits. Finally, in the setting of this rapidly imposed change in care delivery, we sought patient feedback regarding inclusion of telephone and virtual care in their future visits.

BC Children's Hospital is the sole tertiary children's hospital serving British Columbia, Canada. The program has a provincial mandate, and accordingly, the diabetes program serves approximately 900 children with diabetes from across BC. Prior to the COVID-19 pandemic, most patients were seen approximately 1-3 times per year for in-person visits with a multidisciplinary diabetes team. As a result of the COVID-19 pandemic, starting on March 25, 2020 all in-person routine diabetes visits were cancelled, and, in their place, families were provided the opportunity to self-select into either a telephone (audio only) or virtual (video and audio using Skype for Business or Zoom) visit. For telephone visits, the families had a pre-scheduled time for a 30-minute call. For virtual visits, families received an email link for their pre-scheduled 30-minute visit using a computer or smartphone. Prior to each telephone or virtual visit, a diabetes nurse educator contacted the family and encouraged them to send in blood glucose and insulin dose information by email and to upload any diabetes technology data (i.e. pump and/or continuous glucose monitor (CGM)). The nurse also provided connection information for families not linked with our clinic accounts. Files that were received or available in advance of the visit were then printed and available for the clinicians to use during the clinic visit.

A cross-sectional telehealth usability and feedback questionnaire was offered to all families who had a recent telephone or virtual diabetes visit from March 25 to May 27, 2020, and for whom there was an email address available. Based on this inclusion criteria, 301 patient families were identified to be eligible for the study and were invited to participate in the questionnaire via an online link. Survey data were collected from May 28 -June 9, 2020.

The survey was designed and pilot tested by an interdisciplinary team of pediatric endocrinologists, diabetes nurse-and dietitian-educators, endocrinology trainees and diabetes administrative staff. Our survey instrument was also pilot tested with two people living with T1D for comprehension, and ease of completion. Their comments resulted in changes to survey language resulting in increased clarity and readability. The survey consisted of the Telehealth Usability Questionnaire, modified to the current context of telephone and virtual visits. The Telehealth Usability Questionnaire (TUQ) is a validated tool that was designed to assess the delivery of virtual health care. The TUQ addresses multiple aspects of usability: usefulness, ease of use, interface quality, interaction quality, reliability and satisfaction and future use. It was developed with newer-generation virtual care platforms in mind, such as those used by our diabetes team during the COVID-19 pandemic, and it combines items from existing telehealth questionnaires. Questions in the TUQ can be tailored to address participants and virtual health systems in different settings. The TUQ has strong content validity, reliability and it has been noted to be a solid, robust and versatile measure [12] . Of the validated 21-question TUQ, 4 questions for the telephone group and 3 questions for the virtual group were removed as they did not apply in our context. No questions were altered in intention or meaning. Several additional questions were developed for this study to assess the visit characteristics (5 questions) and preferences for future care (6 questions), and these questions were analyzed separately from the TUQ questions. A 4-point Likert scale was used, with response categories of "Not at all/ Partly/ Quite a bit/ Completely", similar to the scale used in the pan-Canadian patient survey [13] . The questionnaire is included as a supplementary file.

Families who had a telephone or virtual diabetes visit from March 25 to May 27, 2020, and for whom there was an email address available, were eligible and were introduced to the study by email. They were provided study information and the opportunity to consent to both survey participation and linkage of clinical information (such as age, sex, time since diagnosis, insulin pump use, CGM use and insulin regimen) with survey results. The introductory email contained a link to complete the survey online. In addition to the initial email, families received a series of 3 automated email reminders over the course of the data collection period.

All families completed the survey via an online database platform, Research Electronic Data Capture (REDCap). REDCap is a secure, web-based application designed to support data capture for Quality Improvement/Quality Assurance (QI/QA) and research studies. REDCap was used to build and administer the survey as well as store survey data.

The BCCH diabetes program follows 890 pediatric patients with T1D or type 2 diabetes mellitus (T2D).

Many of these patients (542/890, 61%) have provided consent to participate in the BC Pediatric Diabetes Registry, a clinical registry that collects de-identified patient-level (e.g. demographic data, diabetes type, co-morbidities, etc.) and visit-level data (e.g. glycated hemoglobin (HbA1c), insulin pump use, CGM use, etc.) for the purpose of supporting quality improvement research. While all patient families with a recent telephone or virtual diabetes visit were invited to participate in the survey, our analysis included only those who were also in the clinical registry, as this allowed for exploration of associations between survey results and clinical characteristics.

The number of visits within the study period and an anticipated response rate of 30% determined the sample size. All participants who completed the survey and who had provided consent to be part of the clinical registry were included in the analysis. Results from the Likert scale were presented in the form of median, interquartile range and percentages as these were found to best describe the central tendency and spread of the data. Tests for statistical significance of outcomes between telephone care and virtual care groups were assessed using Welch's t-test (two-sample assuming unequal variances). The use of t-tests is justified as assumptions of normality of the sample mean have be shown to be robust when applied to Likert scales [14] . Statistical significance was set at P < 0.05. To understand the associations between patient factors and choosing a virtual visit as well as preference to return to all in-person visits, we performed a series of logistic regression models on the outcome of choosing a virtual visit and the outcome of selecting preference for a return to all in-person visits. For both outcomes the following predictors were applied: age at visit, sex, time since diagnosis, pump use, CGM use, HbA1c and distance from BCCH. Both unadjusted and adjusted odds ratios were estimated based on univariable and multivariable logistic regressions. Finally, to compare characteristics of survey respondents to the larger BCCH clinic population, data is presented for available metrics for all patients with T1D or T2D who are participating in the clinical registry. Microsoft Excel 2020™, R version 3.0.1 and SPSS Version 27 were used for analyses.

Approval for the administration of this quality improvement survey was granted by the Research Privacy Advisor (Provincial Health Services Authority), which is the requirement for quality improvement studies at our institution. This study was conducted for quality improvement and monitoring and, therefore, did not fall under the scope of the Research Ethics Board, as per the University of British Columbia Guidance notes, Article 4.4.1 and Tri-Council Policy Statement 2 (TCPS2) Article 2.5 [15, 16] . However, consent was gathered at the time of survey administration, and participants were informed that they could withdraw their consent at any time. Data collection occurred in accordance with the agency's privacy

Tool criteria for Quality Improvement and Evaluation projects [17] .

Out of 301 eligible patient families that were invited via email to participate in the survey, we received 141 completed responses, resulting in a 47% response rate. There were 4 respondents in the telephone care group and 2 respondents in the virtual care group who started but did not complete the survey, and thus were omitted from our analysis. Eighty-seven of the 141 respondents that completed the survey had previously provided consent for their clinical data to be linked via the clinical diabetes registry and were included in the current analysis. The majority of surveys were completed by the parent, family member or guardian of the child with diabetes ( Table 1 ). The characteristics of survey respondents with linked clinical data are provided ( Table 1 ). The telephone care group and virtual care groups were similar in age, time since diagnosis, HbA1c, sex and distance from their diabetes center. Although not statistically significant, there was a higher proportion of pump therapy (57.5% vs 44.7%, P = 0.2) and CGM use (62.5% vs 51.1%, P = 0.3) among families who chose to have their visit by virtual care, compared to the telephone care. We performed a regression analysis to look for associations between patient factors and the choice of visit type (virtual or telephone) as well as preference for returning to in-person visits in the future. There were no significant odds ratios in both the unadjusted and adjusted analysis for both outcomes, although 95% confidence intervals were wide indicating the current study is under-powered to detect an effect (e.g. pump use has an odds ratio of 1.68 (95% CI: 0.72 -3.97)) for opting for a virtual visit.

A comparison between study participants and the diabetes registry with respect to age, sex, type of diabetes, HbA1c, pump use and CGM use, are also included in Table 1 . Overall, the characteristics of the survey respondents were similar to the larger group; however, it is noteworthy that participants of the study, when compared to the greater clinic population, reported higher pump use (50.6% vs 42.3%, P = 0.1) and statistically higher CGM use (56.3% vs 35.8%, P = 0.001).

In place of in-person routine diabetes visits, families were called by our administrative staff and offered the option to self-select into either a telephone (audio only) or virtual (video and audio using Skype for Business or Zoom) visit. For telephone visits, the families had a pre-scheduled time for a 30-minute call.

For virtual visits, families received an email link for their pre-scheduled 30-minute visit using a computer or smartphone. Information about which family members attended the visits is presented in Table 1 , which shows a comparison between the telephone and virtual care groups. Of the 87 complete responses that were included in the current analysis, 47 and 40 responses were from the telephone and virtual care groups, respectively. The proportion of visits including both a family member and the child was much higher for the virtual care group compared to the telephone group. A diabetes doctor attended nearly all visits, while families also reported that a diabetes nurse attended less frequently. Occasionally there was the presence of a dietitian, social worker, medical trainee or other physician.

Overall, across the usability survey components of usefulness, ease of use and learnability, interface quality, interaction quality and satisfaction and future use, all had median scores for both telephone visits and virtual visits of "quite a bit" or "completely", while the scores for reliability for both groups was "partly" ( Table 2 ). The distribution of responses for both groups across the 6 usability components is presented in Figure 1 .

Responses to the individual items on the TUQ generally mirrored those observed from the usability component scores and are outlined in Table 2 . In terms of usefulness, although families believe that a telephone or virtual visit "completely" saves them travel time and "completely" provides for their healthcare needs, they express that the visit only "partly" improves their access to healthcare services. In terms of ease of use and learnability, families reported that the telephone or virtual systems were "completely" simple to use and "completely" easy to learn, although a between-group comparison revealed that the telephone group found it easier to learn the system for the visit than the virtual group (P = 0.005). Overall, usefulness favoured the telephone group (P = 0.005). In terms of interface quality, this also favoured the telephone group (P = 0.013); however, both groups had the same median score of 'quite a bit'. Responses regarding the interaction quality of telephone and virtual visits indicate quite good usability for both, with median scores of "quite a bit" and "completely" across all items. Families reported slightly lower scores for individual questions in both the reliability and satisfaction and future use components. A median score of "partly" was recorded for their perspective on whether the visit was the same as in-person visits, if the visit was an acceptable way to receive healthcare services, and if they would use this service again. However, a higher score of "quite a bit" was reported for families' comfort in communicating with the clinician during the visit, and their overall satisfaction with the visit.

A number of questions were designed to gain insight on family preferences for future telephone and virtual diabetes visits (Table 3 ). There were no notable differences in responses to these questions between the telephone versus virtual care group. For future telephone and virtual visits, almost all families would like the diabetes doctor to be involved, and many would like other members of the multidisciplinary team involved. Families were asked if they would, once the COVID-19 pandemic resolves, still like to continue to receive telephone or virtual visits as part of their diabetes care. Threequarters of families would like telephone or virtual visits to be included in their future care, while about one-quarter of families would prefer all of their future visits to be in-person. A regression analysis was conducted to look for associations between patient factors and the desire to have all in-person future care.

No significant associations were found between potential predictors and in-person care although confidence intervals were wide for some predictors indicating a lack of power with the current sample to identify these associations. 

To our knowledge, this is the first study to report on a tertiary diabetes centre's complete transition to telephone and virtual care for pediatric diabetes during the COVID-19 pandemic. As the only tertiary children's hospital in BC, our program was afforded the opportunity to gain insight from a large number of clinical encounters in a short timeframe during the early stages of the COVID-19 pandemic.

At our centre, when families were able to self-select a telephone or virtual care visit, the overall assessment of the visits by families is positive. The telephone visits were deemed easier to learn to use, and more simple and easy to understand, but overall the ratings of telephone and virtual visits by families were similar across multiple areas on the TUQ. It is interesting that univariate and multivariate modeling revealed no clear patient-related factors associated with families choosing one type of visit over the other.

Furthermore, it stands out strongly that while most participants said that they would like to have more telephone or virtual visits in the future, nearly one quarter of families would like all future care to be inperson. It is essential that we work to identify which families are not compatible with telephone or virtual health visits, so that we may continue to strive for equitable and family-centered care across the province for all. Furthermore, some initial assumptions were not born out in this study: it was not the tech savvy (pump or CGM users) who necessarily chose virtual visits, nor did distance from the diabetes clinic associate with preference for in-person or remote visits. Clearly, we must become more refined in our ability to understand which families will benefit most, or potentially be harmed the most, from a transition to telephone or virtual care.

Our clinic practices family-based care, and accordingly in-person visits are attended by the child/teen and parents/guardians. It is interesting that in the telephone and virtual setting, we notice some visits occurring with the parent/guardian only and no child (19.5%), and a few visits with older teens only and no parent/guardian (4.6%). A much higher proportion of virtual visits had attendance by both child and parent (90%), compared to telephone visits (63.8%). This is interesting for clinicians to consider when scheduling remote visits with families: virtual care, as opposed to telephone care, may facilitate a greater chance of the child participating in the visit and engaging with their own chronic disease management. This is consistent with previous findings that have demonstrated virtual visits facilitate increased engagement and patient and parent involvement in their diabetes care [7, 19] . Additionally, in children and adolescents with T1D, parental care and involvement have been shown to have a positive impact on the patient's diabetes management [20] , which further supports the need to keep families engaged and in close communication with the healthcare team. In times of a pandemic, telehealth is a means to achieve this goal, and here we have shown telephone and virtual visits to have good usability and to be well received by most families. In terms of future visits, many families have a desire for a multidisciplinary approach to phone and virtual visits, including physicians, nurses, dietitians, social workers and interpreters. This is an important family value that we will attempt to incorporate in future care. Virtual health has now become a part of the 'new normal' for provision of sub-specialty pediatrics care in BC during the COVID-19 pandemic. We anticipate that this short-term change to virtual care will transition into a longer-term model of care that, while retaining some aspects of our prior model of care, will also hopefully incorporate the new skills and habits developed by healthcare providers and patient families over the recent months. Given the significant change in our provision of care, we must now ask: which aspects of our telephone or virtual care need to be modified and, eventually, how will we transition to a sustainable model that combines the best features of virtual and in-person care in a way that is safe, effective, and patient-centered. There is much benefit to be had in taking the current opportunity created by the massive uptake of technology in healthcare to rapidly assess for its impact, and to make adjustments in real-time. Our data shows that, on average, families in BC would like 50% of their future pediatric diabetes visits to occur by telephone or virtual technology. Previous work has suggested that telehealth has the potential to support individuals living with T1D in their disease management [21] and that telehealth may be just as effective as in-person visits in maintaining glycemic control and providing diabetes care to the pediatric T1D population [8] . One implication of our findings is that the clinic team at BCCH needs to consider providing telephone and virtual visits as an ongoing option for families, even after the current pandemic is over. This is a dramatic change from our pre-COVID-19 care model, and it opens the door for necessary conversations to be had about space, resources, scheduling and ongoing program evaluation in the development of a new long-term model of pediatric diabetes care in our province.

The strengths of this study are that it captured the perspectives of a large number of families with a good survey response rate, and the respondents represent our overall clinic population. The collective experiences of these participants allow for never-before-captured insight on the transition of a pediatric diabetes clinic to telephone and virtual care in the setting of a pandemic. A limitation is that the perspectives collected are of those who self-selected to participate in the survey, which may introduce responder bias in the dataset. A higher percentage of people were using insulin pump therapy and/or CGM in the survey respondents, when compared to the entire diabetes clinical database group. It is possible that families with more technologically advanced diabetes care may have been more likely to respond to an email request for an online survey. Additionally, this study did not capture the perspectives of the healthcare professionals taking part in these visits. In order to further understand the usability of the telephone and virtual systems in the setting of the current pandemic, this additional group should be considered. Notably, healthcare professionals recently identified a number of benefits in using video consultations for routine pediatric diabetes visits [22] . Additionally, future research should gather more in-depth perspectives through qualitative interviews with patient families and healthcare professionals, which will provide further insight on which families would benefit most and least from telehealth for their diabetes care. Follow-up work will also need to assess for the attainment of standards of care in diabetes practice [18, 23] in the setting of a novel model of routine diabetes care delivery.

We provide the first report on a tertiary diabetes centre's complete transition to telephone and virtual care for routine pediatric diabetes visits during the COVID-19 pandemic. In this setting, where families chose their preferred visit type, both telephone and virtual care visits had impressive usability. A flexible offering of care delivery options should be considered as a potentially helpful approach to providing telehealth care. Furthermore, factors differentiating between families who prefer different styles of telehealth care need to be further delineated. We must strive to understand more about the groups who do, and do not, thrive within a telehealth care setting. This study provides an initial roadmap for how our families wish to see care provided in the future, which appears to be a flexible blend of in-person and telehealth visits, with the option of having a multidisciplinary team participating in the visits. Insights from this study are already informing improved care in the setting of the pandemic within our program.

Ongoing assessment of care provision in the setting of the COVID-19 pandemic, and associated adaptation in care delivery, is essential in a time of such dramatic change. 

COVID-19 outbreak and pediatric diabetes: perceptions of health care professionals worldwide

Caring for Pediatric Patients with Diabetes amidst the Coronavirus Disease 2019 Storm

Missing during COVID-19 lockdown: Children with onset of type 1 diabetes

The impact of COVID-19 on people with diabetes in Brazil

Summary of recommendations regarding COVID-19 in children with diabetes: Keep Calm and Mind your Diabetes Care and Public Health Advice

CoYoT1 Clinic: Innovative Telemedicine Care Model for Young Adults with Type 1 Diabetes

CoYoT1 Clinic: Home Telemedicine Increases Young Adult Engagement in Diabetes Care

Use of Telemedicine to Improve Adherence to American Diabetes Association Standards in Pediatric Type 1

Effect of telehealth system on glycemic control in children and adolescents with type 1 diabetes

Ergonomic requirements for office work with visual display terminals (VDTs) -Part 11: Guidance on usability

A Systematic Review of Pediatric Telediabetes Service Models

Development of the Telehealth Usability Questionnaire (TUQ)

Developing the first pan-Canadian acute care patient experiences survey

Likert scales, levels of measurement and the 'laws' of statistics

UBC Clinical Research Ethics General Guidance Notes | Office of Research Ethics

Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans -TCPS

ARECCI Ethics Guideline Tool

ISPAD Clinical Practice Consensus Guidelines 2018: The delivery of ambulatory diabetes care to children and adolescents with diabetes

Implementing Telehealth in Pediatric Type 1 Diabetes Mellitus

School and pre-school children with type 1 diabetes during Covid-19 quarantine: The synergic effect of parental care and technology

The Silver Lining to COVID-19: Avoiding Diabetic Ketoacidosis Admissions with Telehealth

Assessing the benefits and challenges of video consultations for the treatment of children with type 1 diabetes -A qualitative study among diabetes professionals

Diabetes Canada 2018 Clinical Practice Guidelines for the Prevention and Management of Diabetes in Canada

We thank the families who participated in the survey, as well as the people who participated in survey review and development. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

AF, BH, MI, SA and SZ contributed to study design. AF and BH administered the survey. AA assisted with data collection and data linkage. AF, BH and MI performed data analysis. AF and BH prepared the manuscript. All authors participated in manuscript review and have approved the final version.

No authors have any conflicts of interest to declare.

The way I interact with this telephone/virtual system isThe system is simple and easy to understandThis system is able to do everything I would want it to be able to do

I could easily talk to the