key: cord-0895555-yb0szn5q
authors: Saleem, Jason J; Read, Jacob M; Loehr, Boyd M; Frisbee, Kathleen L; Wilck, Nancy R; Murphy, John J; Vetter, Brian M; Herout, Jennifer
title: Veterans’ Response to an Automated Text Messaging Protocol During the COVID-19 Pandemic
date: 2020-05-29
journal: J Am Med Inform Assoc
DOI: 10.1093/jamia/ocaa122
sha: 4adbadbb0f9e4d26768a89abf97dfc6be3efcde4
doc_id: 895555
cord_uid: yb0szn5q

The US Department of Veterans Affairs (VA) is using an automated short message service (SMS) application named ‘Annie’ as part of their COVID-19 response with a protocol for coronavirus precautions, which can help the Veteran monitor symptoms and can advise the Veteran when to contact their VA care team or a nurse triage line. We surveyed 1,134 Veterans on their use of the Annie application and coronavirus precautions protocol. Survey results support what is likely a substantial resource savings for the VA, as well as non-VA community healthcare. Moreover, the majority of Veterans reported at least one positive sentiment (felt more connected to VA, confident, or educated and/or felt less anxious) by receiving the protocol messages. The findings from this study have implications for other healthcare systems to help manage a patient population during the coronavirus pandemic.

Robust evidence supports the value of integrating SMS/text-messaging interventions into health care delivery and population health practice. 1 The US Department of Veterans Affairs (VA) Office of Connected Care (OCC) developed and implemented an automated short-message service (SMS) application for guided patient self-management, named Annie. 2 Annie can deliver educational and motivational messages to Veterans' mobile phones, send reminders for health tasks, and query about their current health status. This can encourage Veterans to adopt healthy habits and monitor progress toward wellness goals. The flexible design and branching logic allow clinical teams to develop protocols tailored to a variety of health conditions. Annie enables clinical teams to assign protocols to patients based on clinical and social conditions as well as user preferences; they may or may not review responses. In addition, for certain protocols, Veterans can self-enroll. Currently The CPP ( Figure 1 ) is a 2 month protocol that delivers educational content on coronavirus precautions, guides Veterans in monitoring their temperature and other clinical symptoms, and advises them to call their care team or a nurse triage line when they cross certain clinical thresholds. For the first two weeks the protocol was offered, check-ins were sent daily.

Currently, messages include wellness questions sent Monday, Wednesday, Friday, Sunday ("Annie here. Are you feeling well today?") and educational tips (e.g., "Annie again. If you are ill, use a separate bedroom and bathroom if possible.") sent Monday, Wednesday, and Friday.

Annie always identifies herself, e.g., "Annie Here."

Veterans can self-enroll in this protocol or be added by a member of their care team.

OCC reached out to Veterans across the country encouraging them to enroll in Annie and subscribe to the CPP, and around 4200 self-enrolled. As the number grew, we decided to invite CPP subscribers to provide feedback on Annie's utility and value to them during this period.

[ Figure 1 about here]

A subset of the authors designed a set of nine (9) questions to assess the respondents' geographic distribution, self-reported clinical status, reactions to the Annie CPP messages, and actions taken or not taken because of the messages they received. The questions included multiple formats in which the respondents could choose all responses that applied, as well as open-ended questions allowing free-text responses. The questions were loaded into SurveyMonkey and made available via internet link. On April 22, 2020, a broadcast message was sent to 11,630 users of the Annie application who had that feature enabled, which is 81% of all Annie users. At that time, there were approximately 4400 active users of Annie's CPP (96% self enrolled; the other 4% enrolled by a VA staff member). Data was exported from SurveyMonkey, April 23, 2020, 24 hours after the link was sent and transferred to an Excel spreadsheet for analysis. Data were summarized with descriptive statistics.

Questions that included open-ended responses were analyzed using an inductive coding approach 5,6 by a single team member (i.e., allowing codes to emerge from the data rather than using a pre-determined coding scheme) with an independent audit of all coding by a co-author.

This type of auditing procedure by a second analysist is considered an acceptable alternative to using independent coders for ensuring validity of the analysis. 7 Questions raised by the audit were resolved by consensus calls as needed. For two questions, a secondary analysis of the content for each primary code was warranted based on the complexity of the responses to further reveal sub-patterns. For example, for the primary code 'Connected' for the question 'What about Annie's messages was helpful for you?', the first author then summarized the different types of sub-patterns for 'Connected' as sub-patterns under this primary code.

Of the 11,630 Veterans who received an invitation to participate, 1567 (13.5%) went to the SurveyMonkey website. Of those, 188 (12.0%) reported that they were not subscribed to the CPP Annie protocol, so were excluded from analysis. Of the remaining 1379 who were eligible, 245 (17.8%) did not complete the survey, but we retained partial responses; 1134 Veterans responded to all questions. This response was from a total of 4400 Veterans who were actually subscribed to the CPP protocol, so the calculated response rate is 25.8% (1134/4400). 

Frequent free text responses in the 'Other' field for the question 'Why did you choose to subscribe to Annie's Coronavirus Precautions protocol messages?' included reasons such as curiosity, health management, and to simply stay informed.

This was an open-ended question. Table 2 shows a summary of the qualitative analysis conducted on the 808 free text responses. category. These sub-patterns included:  Seven Veterans liked that their status was being checked on. (7)  Six Veterans specifically mentioned that the messages made them feel "not alone".

This open-ended question received 61 responses. Most of those responses were coded as 'Not enough information' (34); that is, they felt that the Annie messages did not provide enough information to be valuable to the user or that they shared repeated, common knowledge. Others reasons included 'Too many messages' (3), the user did not receive a reply from Annie (3), the user did not need the help (3), and the user was limited in how they send messages to Annie (4).

For example, one user was unsure how to respond to the question 'Are you feeling well today?' because "…you can have different ways of not feeling well." Other responses were one-offs or had insufficient information to be coded.

Frequent free text responses in the 'Other' field mostly included users reporting that they were not receiving messages at all or messages that required action of them (13). Other responses had insufficient information in their response to be coded.

The most frequent type of free text response in the 'Other' field included Veterans reporting that they would have sought information from alternative information sources such as the Centers for Disease Control (5), the World Health Organization (1), news outlets (4), a health department (2), medical professionals (2), or the internet in general (6) . Other responses had insufficient information in their response to be coded.

Although most other responses had insufficient information to be coded, a couple users had a critique of the messages, the Annie application, or the process to sign up (2) . Others simply indicated that they use Annie as an information source (3) . Additionally, 879 (76%) of Veterans reported at least one positive sentiment (felt more connected to VA, confident, or educated and/or felt less anxious) as one impact of CPP. This is in line with previous studies on Annie. 9 The over 800 open-ended responses for the question 'What about Annie's messages was helpful for you?' also support the positive sentiment findings ( pandemic.

These findings should be interpreted within the context of certain limitations. First, as guidance is evolving, some changes should be expected to the protocol itself. Around 600

Veterans had opted out of receiving Annie CPP messages prior to our survey and so those

Veterans are not be represented in our sample. However, that is still a small percentage of the active subscribers. Also, we did not collect data on how long each Veteran has been subscribed to the CPP messages. The length of time a Veterans has been receiving the messages may have influenced their responses. Lastly, a couple of free text responses noted that responders were not receiving messages. Since completion was on a "self-selection" basis, it is possible that a few responses were not based on experience with CPP. However, that is true of many survey opportunities. Despite these limitations, the survey seems to suggest that the Annie CPP has had a substantial positive impact on the Veteran population. to "Annie," either as the greeting (e.g., "Annie here") or at the end of the message ("-Annie"). For the wellness check questions, Annie asks for a response and provides predetermined response choices. Figure: messages always include some reference to "Annie," either as the greeting (e.g., "Annie here") or at the end of the message ("-Annie"). For the wellness check questions, Annie asks for a response and provides predetermined response choices.

1449x815mm (72 x 72 DPI)

Mobile text messaging for health: a systematic review of reviews. Annual Review of Public Health

Automated Text Messaging With Patients in Department of Veterans Affairs Specialty Clinics: Cluster Randomized Trial

Internal VA Report: unpublished

Using observational study as a tool for discovery: Uncovering cognitive and collaborative demands and adaptive strategies. How Professionals Make Decisions

A framework for epistemological analysis in empirical (laboratory and field) studies. Human Factors

Physicians' beliefs about using EMR and CPOE: in pursuit of a contextualized understanding of health IT use behavior

Internal VA Report: unpublished

Use of Mobile Messaging System for Self-Management of Chemotherapy Symptoms in Patients with Advanced Cancer

The authors thank Ralph Strenglein, Tim Hogan, Amanda Sheckman, Kelly Ehasz, Treva