key: cord-0835776-uu65a918 authors: Tai‐Seale, Ming; May, Nicole; Sitapati, Amy; Longhurst, Christopher A. title: A learning health system approach to COVID‐19 exposure notification system rollout date: 2021-10-13 journal: Learn Health Syst DOI: 10.1002/lrh2.10290 sha: e9f8b3110984c50861276d6ac9318372c188f250 doc_id: 835776 cord_uid: uu65a918 INTRODUCTION: Digital exposure notification (EN) approaches may offer considerable advantages over traditional contact tracing in speed, scale, efficacy, and confidentiality in pandemic control. We applied the science of learning health systems to test the effect of framing and digital means, email vs Short Message Service (SMS), on EN adoption among patients of an academic health center. METHODS: We tested three communication approaches of the Apple and Google EN system in a rapid learning cycle involving 15 000 patients pseudorandomly assigned to three groups. The patients in the first group received a 284‐word email that presented EN as a tool that can help slow the spread. The patients in the second group received a 32‐word SMS that described EN as a new tool to help slow the spread (SlowTheSpreadSMS). Patients in the third group received a 47‐word SMS that depicted the system as a new digital tool that can empower them to protect their family and friends (EmpowerSMS). A brief four‐question anonymous survey of adoption was included in a reminder message sent 2 days after the initial outreach. RESULTS: One hundred and sixty people responded to the survey within 1 week: 2.33% from EmpowerSMS, 0.97% from SlowTheSpreadSMS, and 0.53% from emails; 29 (41.43%), 24 (41.38%), and 11 (34.38%) reported having adopted EN from each group, respectively. Patient reported barriers to adoption included iOS version incompatibility, privacy concerns, and low trust of government agencies or companies like Apple and Google. Patients recommended that healthcare systems play an active role in disseminating information about this tool. Patients also recommended advertising on social media and providing reassurance about privacy. CONCLUSIONS: The EmpowerSMS resulted in relatively more survey responses. Both SMS groups had slightly higher, but not statistically significant EN adoption rates compared to email. Findings from the pilot not only informed operational decision‐making in our health system but also contributed to EN rollout planning in our State. Numerous efforts aim at using smartphones emitting randomized Bluetooth codes to notify people exposed to COVID-19-positive individuals. These digital exposure notification (EN) approaches may offer considerable advantages over traditional contact tracing in speed, scale, efficacy, and confidentiality in pandemic control. 1 With privacy concerns being a significant barrier to the adoption of traditional contact tracing and EN, 2 it is essential to effectively communicate with the public the purpose of contact tracing and why a digital EN tool is being provided. While government and technology companies have made EN tools available to the public, academic health centers, as hubs of innovation and learning health systems, [3] [4] [5] have an important role to play in improving the adoption of these tools. For this reason, UC San Diego Health, an academic health center with a strong history of digital innovation, 6, 7 helped to lead the pilot of the EN tool for the State of California. This brief study reports a rapid-cycle learning project testing the wording of messages, and the digital means with which they were sent, email vs Short Message Service (SMS), aimed at encouraging adoption among UC San Diego Health patients. Behavioral economics literature provides evidence that the framing of messages can influence behavior. 8, 9 Based on evidence that many people feel powerless in the pandemic 10, 11 and that gain framing messages could promote risk-aversion and preventive behaviors, 12 we developed a message about EN as a new tool that empowers people to protect their family and friends. This empowerment-framing message was expected to promote pro-social and prevention behavior. 11, 13, 14 We were interested in testing the effectiveness of the empowerment-framing message in comparison with two other messages conveying EN as a tool for slowing the spread of COVID-19, but without mentioning, it as an empowerment tool. We hypothesized that the empowerment-framing SMS message would result in a higher rate of adoption of a digital EN tool among patients. We had a week to conduct the pilot project before contacting all 350 000 of our patients prior to the Thanksgiving Holiday when people were at higher risk of being exposed to COVID. Our intention was to apply the lessons learned from the pilot to facilitate the rollout of EN to our entire patient population and then facilitate EN adoption in California. Commonly known as A/B testing, 15 the pilot project took place at the UC San Diego Health (UCSDH) System. We tested three communication approaches of the Apple and Google EN tool (CA Notify) in our patient population in a rapid learning cycle. 3 The Institutional Review Board certified the project as not qualifying as human subject research. We contacted 15 000 UCSDH patients for the initial EN pilot on November 17, 2020, followed by a reminder and a survey on November 19, 2020. Patients were eligible and automatically included on the active patient registry if they had a face-to-face encounter with a UC San Diego Health provider in the past 36 months, including televideo visits, age 18 years or older. The 15 000 patients in the pilot were initially informed of the EN tool in three separate approaches after their medical records number (MRN) were pseudorandomly 3 assigned to three groups. The first group included the first 6000 patients with an even MRN. The second group included the first 6000 patients with an odd MRN. A third group included the next 1500 patients with even MRNs and the next 1500 patients with odd MRNs, after the first two groups had been formed. Patients in the first group received an email that contained 284 words and presented CA Notify as a tool that can help slow the spread. The patients in the second group received a 32-word Short Message Service (SMS) that described the EN system as a new tool to help slow the spread (SlowTheSpreadSMS). The patients in the third group received an SMS with 47 words that described the system as a new digital tool that can empower them to protect their family and friends (EmpowerSMS) (see Appendix S1 for each message.) A four-question anonymous survey was sent to all 15 000 patients in the pilot as part of a reminder message 2 days later, on November 19, 2020. The same communication approach as the initial outreach was used, that is, email reminder to the email group and SMS reminders to the two SMS groups. The survey asked whether they had adopted the EN tool and recommended the tool to other people, reasons for non-adoption if they had not adopted it, and their recommendations for how UCSDH could improve its communication about the tool so more people would adopt CA Notify. Rates of adoption were tabulated from survey responses in each group and compared across the three groups using logistic regression analysis. Free text responses to reasons for non-adoption and recommendations for improvements were coded and analyzed iteratively and thematically. 16 4 | RESULTS From 15 000 reminders with invitation to survey, 160 people responded within 1 week, 2.33% from EmpowerSMS, 0.97% from SlowTheSpreadSMS, and 0.53% from the email group. Among the respondents, 29 (41.43%), 24 (41.38%), and 11 (34.38%) reported having adopted CA Notify from each group, respectively (Table 1) . These rates were not statistically different from each other. On average, 34% had recommended the tool to others (not tabulated). Data suggested that 32 patient respondents reported getting an error message that the tool was not available in California after they tried to activate the tool (see Section 4.2 for more descriptions.) This finding implied that these patients had attempted to activate the tool but were not able to complete the task because their iPhones had older iOS versions. Therefore, the potential adoption rates would have been considerably higher than the observed adoption rates had the EN tool been accessible to iPhones running on older iOS versions. Some patients expressed appreciations for being informed of the EN tool by us. For example, one patient wrote, "You found and influenced me. I appreciate that." As mentioned above, answers to the question on reasons for non-adoption suggested that many patients had tried to turn on CA Notify but got an error message that the program was not available in California. We learned that the error was because their iPhones did not have iOS14.2, which was required at the time of the pilot. Besides the iOS version incompatibility, reasons for non-adoption included privacy concerns, low trust of government or Apple and Google, lack of a smartphone, help need, not receiving initial outreach, not knowing it was available, and misunderstandings about having their location tracked by EN (Table 2) . The EmpowerSMS group had a slightly higher adoption rate than the SlowTheSpreadSMS group and the email group. The differences were not statistically significant, however. A relatively higher proportion of patients in the EmpowerSMS group responded to our survey, compared to those in the other groups. It is plausible that the empowerment gain frame had nudged more patients to interact with our survey. The results from 160 survey respondents suggested that 41% had adopted EN after receiving an empowerment-framing SMS, with more people intended to adopt but had encountered difficulties due to iOS versions. Furthermore, 34% of those who had adopted the tool rec- A low number of responses from patients were received making the findings preliminary. The decision to keep the survey anonymous and brief confined the learning to what is available from the survey alone. Furthermore, the contracted digital health firm was not able to provide us with data on message open rates, which limited our ability to measure the more immediate indicator of the effectiveness of our approaches, that is, getting the recipient to read our message and go to our website. Future outreach efforts should strengthen use data capturing while preserving user privacy. We further acknowledge that the reliance on smartphones excludes those who do not own smartphones and those who own smartphones but do not have the technical ability to opt in. A technical support phone line provides assistance to callers in English and Spanish. For users whose smartphones were too old to access the EN system at the time of the pilot, they were able to access EN after December 14, 2020, if their phones use iOS12.5 or later. Relatedly, EN is offered in 13 languages so that language barrier to EN is less likely. Lastly, issues related to civil liberties and corporate or government power associated with digital EN are important but out of scope for this brief research article. This pilot was designed to allow us to rapidly incorporate real patient feedback into the way we communicated the use and benefits of the digital EN tool. Applying the science of learning health systems 5 through theory-guided rapid-cycle testing and dissemination of promising communication approaches, the pilot project team not only informed fast-paced operational decision-making in our health system but also contributed to this important public health effort in California. Quantifying SARS-CoV-2 transmission suggests epidemic control with digital contact tracing Key findings about Americans' views on COVID-19 contact tracing Creating a learning health system through rapid-cycle, randomized testing Transforming from centers of learning to learning health systems: the challenge for academic health centers The science of learning health systems: foundations for a new journal Rapid response to COVID-19: health informatics support for outbreak management in an academic health system Technology-enabled consumer engagement: promising practices at four health care delivery organizations Applying insights from behavioral economics to policy design Psychological and behavioral response to the coronavirus (COVID-19) pandemic Prospect Theory in Times of a Pandemic: The Effects of Gain versus Loss Framing on Policy Preferences and Emotional Responses During the 2020 Coronavirus Outbreak: Evidence from the US and the Netherlands Prospect theory: an analysis of decision under risk Using insights from behavioral economics to mitigate the spread of COVID-19. Appl Health Econ Health Policy Health message framing effects on attitudes, intentions, and behavior: a meta-analytic review Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing Constructing Grounded Theory: A Practical Guide through Qualitative Analysis Do we need a contact tracing app? SUPPORTING INFORMATION Additional supporting information may be found in the online version of the article at the publisher's website. How to cite this article Christopher Longhurst and Nicole May had a contract with the California Department of Public Health to support exposure notification for California.