key: cord-0426892-a55a36vr
authors: Thorpe, A.; Fagerlin, A.; Butler, J.; Stevens, V.; Drews, F. A.; Shoemaker, H.; Riddoch, M. S.; Scherer, L. D.
title: Communicating about COVID-19 vaccine development and safety
date: 2021-07-02
journal: nan
DOI: 10.1101/2021.06.25.21259519
sha: 8ea431c09ab10738b35c4c651afe64a9e4838efe
doc_id: 426892
cord_uid: a55a36vr

Purpose. Beliefs that the risks from the vaccine outweigh the risks from getting COVID-19 and concerns that the vaccine development process was rushed and lacking rigor have been identified as important drivers of hesitancy and refusal to get a COVID-19 vaccine. We tested whether messages designed to address these beliefs and concerns might promote intentions to get a COVID-19 vaccine. Method. An online survey fielded between March 8-March 23, 2021 with US Veteran (n=688) and non-Veteran (n=387) respondents. In a between-subjects experiment, respondents were randomly assigned to a control group (with no message) or to read one of two intervention messages: 1. a fact-box styled message comparing the risks of getting COVID-19 compared to the vaccine, and 2. a timeline styled message describing the development process of the COVID-19 mRNA vaccines. Results. Most respondents (60%) wanted a COVID-19 vaccine. However, 17% expressed hesitancy and 23% did not want to get a COVID-19 vaccine. The fact-box styled message and the timeline message did not significantly improve vaccination intentions, F(2,359)=0.91, p=.402, partial eta squared=.005, or reduce the time respondents wanted to wait before getting vaccinated, F(2,307)=0.76, p=.468, partial eta squared=.005, compared to no messages. Discussion. We did not find an impact on vaccine intention based on providing information about vaccine risks and development. Further research is needed to identify how to effectively address concerns about the risks associated with COVID-19 vaccines and the development process and to understand additional factors that influence vaccine intentions.

All of the currently available COVID-19 vaccines have demonstrated high efficacy against COVID-19 (Olliaro, 2021) . Maximizing the public health benefits from these vaccines depends on achieving high levels of vaccine coverage. One of the major barriers to achieving widespread coverage of COVID-19 vaccines is public hesitancy and reluctance to receive them (Sallam, 2021) . Prior research has established that attitudes towards vaccines and intentions to receive or refuse them are driven by a multitude of factors (Larson et al., 2014) . Within this literature, there is consistent evidence that people who are hesitant or reluctant to receive a vaccine often have doubts about the benefits of vaccines and concerns about the general safety of taking them (Karafillakis & Larson, 2017) .

Throughout the pandemic, how people perceive the threat posed by COVID-19 has been shown to vary based on a number of factors such as age (Bruine de Bruin, 2020) , political ideology (Calvillo et al., 2020; Kerr et al., 2021) , and philosophical beliefs (Byrd & Białek, 2021) . In extreme cases, some people believe that COVID-19 does not pose a threat at all as they question the legitimacy of the pandemic and deny the existence of the virus (Gorski, 2020) . How people perceive the risk of COVID-19 matters as those who do not perceive COVID-19 to be a serious threat are likely to undervalue the benefits of vaccination (Karafillakis & Larson, 2017) and are therefore less likely to want to receive a vaccine (Betsch, et al., 2018; Malik et al., 2020) . Furthermore, people who do not consider COVID-19 to present a serious threat may also be reluctant to receive a vaccine because they are more likely to perceive that they are more at risk from potential harms from vaccination than from acquiring COVID-19 (Karafillakis & Larson, 2017; Betsch, et al., 2018) .

The perceived safety of a vaccine is another factor that can have a strong influence on peoples' decisions about whether or not to get vaccinated (Karafillakis & Larson, 2017; Karlsson et al., 2021) . Current evidence suggests that people who are hesitant about receiving a COVID-19 vaccine are often concerned about the safety of the COVID-19 vaccine development process (Dror et al., 2020) . Producing a number of highly effective vaccines within a year of an infectious disease outbreak is undoubtedly one of humanity's greatest medical achievements. It is important to acknowledge that this achievement was only possible because of decades of prior research and clinical trials on vaccine technology (e.g., mRNA and viral vectors) and previous coronavirus outbreaks (e.g., SARS and MERS) (Ball, 2020; Li et al., 2020) . However, the importance and rigor of this prior research has often been overlooked in public discourse, which has more commonly emphasized the speed of development. As a consequence, concerns that the safety of the vaccines was compromised by rushed development and the use of experimental or untested technology have become widespread (Dror et al., 2020) . High-profile media coverage of very rare side-effects following COVID-19 vaccination (e.g., anaphylaxis and thrombosis with thrombocytopenia syndrome) may have compounded this issue by disproportionately raising public concern about vaccine safety and further fueling public hesitancy (Dodd et al., 2021; Tran et al., 2018) . In addition, it is also possible that the emergency use authorization (EUA) given to the COVID-19 vaccines in response to the severity of the COVID-19 pandemic also resulted in concerns about the safety of the vaccines and, in turn, increased public hesitancy (Kreps et al., 2020; Quinn et al., 2009) .

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted July 2, 2021. ; https://doi.org/10. 1101 In summary, people who are hesitant about receiving a COVID-19 vaccine may believe that the risks from the vaccine outweigh the risks from getting COVID-19, or have concerns about the safety of the vaccine development process. Providing information that addresses these concerns may result in people being more willing to get a COVID-19. Thus, designing and testing communications to address these concerns represents an important goal for research. In the present study we tested whether messaging about the comparative risks of getting COVID-19 vs. the risk of COVID-19 vaccines, and also messaging about the timeline of mRNA vaccine development, might promote intentions to get a COVID-19 vaccine.

Study population and recruitment. Respondents were recruited by Qualtrics Online Panels between March 8-March 23, 2021. This experiment was conducted in the third wave of a threewave longitudinal study. The study was administered online (in English) and was approved by the relevant IRBs. Respondents were compensated for their participation based on the terms of their panel agreement.

A total of 1075 respondents completed the third wave of the study, of which 688 (64%) were United States Veterans; median age of the sample was between 55 and 74 years old; 841 respondents (78%) were male, 819 respondents (76%) were non-Hispanic White, and the median household income was $70,000-$99,999. Sample retention from the first wave was 52% overall, 65% for Veteran and 38% for non-Veterans. There were 361 (33%) respondents who reported having not received any vaccine doses, 243 (23%) who reported having received 1 dose, and 471 (44%) who reported having received 2 doses. Respondents who reported having received at least 1 dose of a COVID-19 vaccine were not included in the experimental component of the survey, leaving a final sample of 361 respondents (Table 1) .

In a fully between-subjects experiment, respondents saw either a Schwartz et al. (2007) factbox style message about the risks of getting a vaccine compared to getting COVID-19 (factbox message), a message about the development of COVID-19 vaccines (timeline message; both messages are presented in Figure 1 ), or no message (control group). Respondent assignment to study conditions was carried out using the built-in randomizer function in Qualtrics' survey flow.

Respondents then reported their COVID-19 vaccine intentions ("When a coronavirus vaccine becomes available to you, how interested are you in getting the vaccine?") using a 5-point scale (1="I definitely do NOT want the vaccine", 2= "Less than one month", 3="Unsure", 4="I WANT to get the vaccine", 5="I definitely WANT the vaccine") and how long they intended to wait before getting vaccinated ("How soon after the COVID-19 vaccine becomes available to you would you become vaccinated?") using an 8-point scale (1="Immediately", 2="Less than one month", 3="One month to less than 3 months", 4="3 months to less than six months", 5="6 months to less than 1 year", 6="1 year to less than 2 two years", 7="I would wait 2 years or more", 8="I would never get it").

We also asked respondents about their views regarding COVID-19 vaccine safety ("In your view, how safe is the COVID-19 vaccine?") using a 5-point scale (1="Not at all safe", 2= "Slightly safe", 3="Somewhat safe", 4="Moderately safe", 5="Extremely safe") and the . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 2, 2021. ; https://doi.org/10.1101/2021.06.25.21259519 doi: medRxiv preprint possibility of experiencing side effects ("How worried are you about experiencing side effects from the COVID-19 vaccine?" using a 5-point scale (1="Not at all concerned", 2= "Slightly concerned", 3="Somewhat concerned", 4="Moderately concerned", 5="Extremely concerned").

Pre-registered analyses. We ran two omnibus one-way ANOVA analyses to examine whether the COVID-19 vaccine risk comparison factbox message and vaccine development timeline message influenced 1) respondents' intentions to get vaccinated and 2) the time they would wait to get vaccinated. Analyses were repeated controlling for age and gender.

We ran two additional omnibus one-way ANOVAs to examine whether 1) the factbox message comparing the risks associated with getting COVID-19 to the risks associated with receiving a COVID-19 vaccine reduced respondents' concern about experiencing side effects from the COVID-19 vaccine and 2) whether the timeline message regarding the vaccine development process increased respondents' views about how safe the COVID-19 vaccine is. All analyses were performed using RStudio statistical software (RStudio Team, 2021) .

Contrary to our expectations, we found no differences between groups regarding vaccination intentions, F(2,358)=0.86, p=.425, ߟ ଶ =.005, or the time respondents would wait to get vaccinated, F(2,306)=0.79, p=.453, ߟ ଶ =.005. In fact, as shown in Figure 2 , mean scores for both the intervention groups suggest that respondents were slightly less willing to get a COVID-19 vaccine and also would want to wait longer before getting one compared to those in the control group who did not see any message.

These results did not differ when controlling for respondents' age, (no effect of group on vaccination intentions, F(2,355)=0.06, p=.945, ߟ ଶ <.001, and no effect of group on time respondents would wait to get vaccinated, F(2,303)=0.05, p=.950, ߟ ଶ <.001), or gender, (no effect of group on vaccination intentions, F(2,352)=1.46, p=.233, ߟ ଶ =.008, and no effect of group on time respondents would wait to get vaccinated, F(2,301)=1. 54 =.010, did not differ between groups. Again, these findings were contrary to our initial prediction.

Exploratory analyses and results. We conducted exploratory analyses using omnibus one-way ANOVAs to examine the influence of the messages on vaccine intentions and time to getting vaccinated on two subsets of the total sample. These analyses were not planned a-priori, but were conducted after the unexpected finding that neither of the messages had an impact on respondents' vaccination intentions or the time they wanted to wait before getting vaccinated.

To examine the possibility that any impact of the messages on vaccination intentions could be dependent on whether respondents took the time to read and process the information provided in them, we re-ran our analyses on a subset of respondents who spent at least 10 seconds on the page displaying the intervention messages (n=298). We found that results from this subset of respondents did not differ from the overall sample: there were no significant differences in . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 2, 2021. Another possibility is that any impact of the messages on vaccination intentions may be limited only to respondents who were initially hesitant about getting a COVID-19 vaccine (Freeman, et al. 2021) . That is, an effect of the intervention might be dampened by ceiling effects on vaccination intentions. For the second subset we only selected respondents who had reported that they were hesitant about getting a COVID-19 vaccine in an earlier survey fielded in December 2020 (i.e., reported that they either "do NOT want to get the vaccine" or "definitely do NOT want to get the vaccine"). Although this subset included a small number of participants, there were again no significant differences between groups on respondents' vaccination intentions, F(2,83)=0.32, p=.724, ߟ ଶ =.008, or the time they would wait to get vaccinated, F(2,61)=1.05, p=.356, ߟ ଶ =.033.

Major advances in vaccine development are at risk of being undermined by increasing levels of vaccine hesitancy (WHO, 2019). In the immediate term, public hesitancy towards COVID-19 vaccines presents a notable barrier to stopping the spread of COVID-19 and ending the pandemic (Sallam, 2021) . Messaging designed to address prominent concerns about COVID-19 vaccines might encourage people who are otherwise hesitant about getting vaccinated to receive one.

In the present study we found no evidence that the messages comparing the potential risks of getting COVID-19 to the potential risks associated with COVID-19 vaccines and regarding the vaccine development process, had any effect on respondents' intentions to get a COVID-19 vaccine, the time that they would want to wait before getting vaccinated, perceived safety of COVID-19 vaccines, or worry about vaccine-related side effects. These results were unanticipated and remained consistent when controlling for the age and gender of respondents and following exploratory analyses of respondents who spent at least 10 seconds looking at the messages and those who had previously expressed that they did not want to get a COVID-19 vaccine. The messages were designed to address concerns about the risks from COVID-19 vaccines and the vaccine development process as they are reasons frequently cited by people who are hesitant about receiving a COVID-19 vaccine (Dodd et al., 2021; Dror et al., 2020) . The present findings are unfortunate as effective low-cost messaging strategies could contribute substantially to combatting growing levels of vaccine hesitancy and help end the COVID-19 pandemic.

One potential reason for these null findings could simply be that the information in the two messages was not sufficiently convincing to influence the vaccine intentions of people who are concerned about the risks of vaccines and the development process. For individuals who did not have these concerns, it could also have been the case that the information in the messages may have drawn their attention to these issues which may have then served to undermine their prior confidence in vaccine development and safety (Betsch et al., 2011) . It is unlikely that these null effects were due to insufficient statistical power. Observed effect sizes in this experiment were very small (such that even if significant, the meaningfulness of these differences would have . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 2, 2021. ; https://doi.org/10.1101/2021.06.25.21259519 doi: medRxiv preprint been questionable), and if anything, the intervention groups showed slightly more vaccine hesitancy than the control group, not less.

Given the cross-sectional nature of this experiment, it is not possible to examine whether repeated exposure to these messages may have yielded more positive results. Repeated messaging about health issues is often more effective at changing attitudes and behaviors (Frew, et al., 2014) , however, given the prevalence of media coverage about COVID-19 and vaccine development it is also possible that efficacy of these messages may be suppressed by respondents having high prior exposure to similar information.

It is also possible that the design of the message-regarding both the style and content, may have also contributed to the null finding. However, given the data available from the present study it is not possible to posit informed justifications for the null findings based on respondents' perceptions of the messages themselves. This does prompt an avenue for future research, which might leverage qualitative methods (e.g., focus groups or open responses) in order to better understand how people perceived the messages used in the present study.

The present findings are aligned with research which suggests that providing corrective information based on an information-deficit assumption is not always effective at addressing people's concerns and doubts (Dubé et al., 2015; Nyhan et al., 2014; Nyhan & Reifler, 2015) . The present findings also highlight the need for testing public health messages to ensure they are as effective as possible. Future research might explore whether different message styles or sources (e.g., non-partisan experts or popular figures) may be more effective at sufficiently addressing concerns about vaccine development and safety. We also believe these findings are important in highlighting the need for research into alternative influences of vaccine intentions that might be more receptive to messaging than concerns about safety and development.

Despite evidence that self-reports are good predictors of health behaviors (Webb & Sheeran, 2006) , it is important that these findings are considered in the context of known limitations of this method (e.g., social desirability). The study design (i.e., US-based recruitment, online, and in English) prevents generalization of the present findings outside of the US and to people who may have limited internet access and lower English proficiency. Furthermore, our sample consists of both Veteran and non-Veteran respondents completing the third (and final) survey of a longitudinal study, which began in December, 2020 and thus are not representative of the general population.

In the present study, providing corrective information about the risks of getting COVID-19 compared to receiving a COVID-19 vaccine and about the development of COVID-19 vaccines was not effective at promoting intentions to get a COVID-19 vaccine. Further research is needed to identify how to effectively address concerns about the risks of side effects from COVID-19 vaccines and the vaccine development process.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 2, 2021. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 2, 2021. to promote the benefits of vaccines. https://www.who.int/news/item/12-09-2019vaccination-european-commission-and-world-health-organization-join-forces-topromote-the-benefits-of-vaccines . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted July 2, 2021. ; https://doi.org/10.1101/2021.06.25.21259519 doi: medRxiv preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 2, 2021. ld . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted July 2, 2021. ; https://doi.org/10.1101/2021.06.25.21259519 doi: medRxiv preprint

COVID-19 VACCINE MESSAGING compliance during the COVID-19 pandemic

Political Ideology Predicts Perceptions of the Threat of COVID-19 (and Susceptibility to Fake News About It)

Concerns and motivations about COVID-19 vaccination

Vaccine hesitancy: The next challenge in the fight against COVID-19

Strategies intended to address vaccine hesitancy: Review of published reviews

Effects of different types of written vaccination information on COVID-19 vaccine hesitancy in the UK (OCEANS-III): a single-blind, parallel-group, randomised controlled trial. The Lancet Public Health

Factors associated with maternal influenza immunization decision-making: Evidence of

Determinants of COVID-19 vaccine acceptance in the US. EClinicalMedicine, 26, 100495

Does correcting myths about the flu vaccine work? An experimental evaluation of the effects of corrective information

Effective Messages in Vaccine Promotion: A Randomized Trial

What does 95% COVID-19 vaccine efficacy really mean?

Public Willingness to Take a Vaccine or Drug Under Emergency Use Authorization during the

RStudio: Integrated Development Environment for R. RStudio

COVID-19 vaccine hesitancy worldwide: A concise systematic review of vaccine acceptance rates

The Drug Facts Box: Providing Consumers with Simple Tabular Data on Drug Benefit and Harm

Acknowledgments. The views expressed in this paper are those of the authors and do not