key: cord-0951311-mphd4z49 authors: Torres, Carlos; Ogbu-Nwobodo, Lucy; Alsan, Marcella; Stanford, Fatima Cody; Banerjee, Abhijit; Breza, Emily; Chandrasekhar, Arun G.; Eichmeyer, Sarah; Karnani, Mohit; Loisel, Tristan; Goldsmith-Pinkham, Paul; Olken, Benjamin A.; Vautrey, Pierre-Luc; Warner, Erica; Duflo, Esther title: Effect of Physician-Delivered COVID-19 Public Health Messages and Messages Acknowledging Racial Inequity on Black and White Adults’ Knowledge, Beliefs, and Practices Related to COVID-19: A Randomized Clinical Trial date: 2021-07-14 journal: JAMA Netw Open DOI: 10.1001/jamanetworkopen.2021.17115 sha: 3fe9453d08d4acd1c62844f30242eb83a38f3071 doc_id: 951311 cord_uid: mphd4z49 IMPORTANCE: Social distancing is critical to the control of COVID-19, which has disproportionately affected the Black community. Physician-delivered messages may increase adherence to these behaviors. OBJECTIVES: To determine whether messages delivered by physicians improve COVID-19 knowledge and preventive behaviors and to assess the differential effectiveness of messages tailored to the Black community. DESIGN, SETTING, AND PARTICIPANTS: This randomized clinical trial of self-identified White and Black adults with less than a college education was conducted from August 7 to September 6, 2020. Of 44 743 volunteers screened, 30 174 were eligible, 5534 did not consent or failed attention checks, and 4163 left the survey before randomization. The final sample had 20 460 individuals (participation rate, 68%). Participants were randomly assigned to receive video messages on COVID-19 or other health topics. INTERVENTIONS: Participants saw video messages delivered either by a Black or a White study physician. In the control groups, participants saw 3 placebo videos with generic health topics. In the treatment group, they saw 3 videos on COVID-19, recorded by several physicians of varied age, gender, and race. Video 1 discussed common symptoms. Video 2 highlighted case numbers; in one group, the unequal burden of the disease by race was discussed. Video 3 described US Centers for Disease Control and Prevention social distancing guidelines. Participants in both the control and intervention groups were also randomly assigned to see 1 of 2 American Medical Association statements, one on structural racism and the other on drug price transparency. MAIN OUTCOMES AND MEASURES: Knowledge, beliefs, and practices related to COVID-19, demand for information, willingness to pay for masks, and self-reported behavior. RESULTS: Overall, 18 223 participants (9168 Black; 9055 White) completed the survey (9980 [55.9%] women, mean [SD] age, 40.2 [17.8] years). Overall, 6303 Black participants (34.6%) and 7842 White participants (43.0%) were assigned to the intervention group, and 1576 Black participants (8.6%) and 1968 White participants (10.8%) were assigned to the control group. Compared with the control group, the intervention group had smaller gaps in COVID-19 knowledge (incidence rate ratio [IRR], 0.89 [95% CI, 0.87-0.91]) and greater demand for COVID-19 information (IRR, 1.05 [95% CI, 1.01-1.11]), willingness to pay for a mask (difference, $0.50 [95% CI, $0.15-$0.85]). Self-reported safety behavior improved, although the difference was not statistically significant (IRR, 0.96 [95% CI, 0.92-1.01]; P = .08). Effects did not differ by race (F = 0.0112; P > .99) or in different intervention groups (F = 0.324; P > .99). CONCLUSIONS AND RELEVANCE: In this study, a physician messaging campaign was effective in increasing COVID-19 knowledge, information-seeking, and self-reported protective behaviors among diverse groups. Studies implemented at scale are needed to confirm clinical importance. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT04502056 [welcome] Welcome to our study! First, we will ask you a few questions. Then, we will show you videos recorded by Medical Doctors. Finally we will ask some more questions. It is very important that you pay close attention to the information in this study. We hope you find it interesting! [persons_met] Think about the most recent weekday. How many different people outside your household did you interact with at a distance of less than 6 feet? o None (1) [attn_check_color] It is important that we know you pay attention to this study. Please enter the word puce below when prompted for your favorite color. [wtp_masks] In this question we ask you how much a pair of reusable cloth masks is worth to you. Some individuals will be chosen to get a coupon worth 2 reusable cloth masks, made from a breathable, stretchable fabric and available in many popular designs. Some others will get an Amazon gift card. We explain how the prizes will be awarded below. Do not worry about the details of this reward system. It is in your best interest to simply report how much a pair of reusable cloth masks is worth to you -how much you would be willing to pay for them in a store. $0 $20 $40 How much US$ are 2 reusable masks worth to you? [Q167 WTP Instructions displayed underneath slider] -You will first tell us how much you would pay for two reusable masks. -A lottery will decide if you receive any prize or coupon. The probability to be chosen is 1/1000. -If you are selected, the computer will choose a random prize value between $0 and $40 (uniformly). -If the prize value is equal to or above your stated value for the masks, you will get an Amazon gift card equal to this prize amount. -If the prize value is less than your stated value, you will get a coupon that can be redeemed for 2 reusable masks at an online store. [ Respondents watched a total of four videos. The scripts for each video are reproduced below. Introductory AMA Racism Statement (Intervention): The American Medical Association recognizes that racism in its systemic, structural, institutional, and interpersonal forms is an urgent threat to public health, the advancement of health equity, and a barrier to excellence in the delivery of medical care. The American Medical Association opposes all forms of racism. The American Medical Association denounces police brutality and all forms of racially-motivated violence. The American Medical Association will actively work to dismantle racist and discriminatory policies and practices across all of health care. The American Medical Association believes in transparency in prescription drug pricing, and we are pleased the House Ways & Means Committee moved the issue forward. Patients and their physicians want to be armed with more information, yet the current situation is opaque if not impenetrable. The committee is rightfully determined to expose factors that lead to high drug prices, and we look forward to continuing our efforts in that regard. Although many people who get sick from COVID will get better, some people who get it become very ill and some even die. Although there's no cure, there are ways medical professionals have found to protect you and your community from COVID. I hope that this message can give you information that will help you protect you or someone you love from COVID infection. First, I would like to tell you about the symptoms of COVID-19. The most common symptoms of COVID-19 are cough, fever, and trouble breathing. Another odd symptom some people have is loss of taste or smell. A large number of people who have COVID-19 actually don't show any symptoms at all. Unfortunately, people can still spread the disease to others even with no symptoms. The next video will provide you with more information on how you can protect yourself and others. Most adults need to sleep between 6 and 8 hours a night. Now, there are some people who get five hours a night and they are fine, so there is some variation across people. But for most adults, we need 6 to 8 hours in order to function well the next day. If you feel sleep deprived you might not be able to function as well as you would normally like. It's important to have something called sleep hygiene which is a routine you follow at bedtime and can help you fall asleep. Things that can disrupt sleep hygiene include caffeine or alcohol too close to bedtime. Eating late at night can also cause indigestion. So keep a routine and trying to get 6-8 hours is important. Americans and other minority groups are three times as likely to get and, when you account for age, four times as likely to die from COVID as white Americans. Without a safe and effective vaccine or therapy, our only option is to continue taking precautionary measures to protect ourselves, our communities, and the most vulnerable among us. While there is no way to ensure zero risk of infection from COVID-19, observing these three practices will help to protect you and others. First, continue to practice social distancing whenever possible: Try to stay outdoors, and to the maximum extent possible, please stay 6 feet apart. If you must be indoors, use visual reminders-like signs, chair arrangements, markings on the floor, or arrows-to help remind you to keep your distance from others, and maintain physical barriers whenever possible. Second, continue to wash your hands often for at least 20 seconds with soap and water, especially before going out, and every time you return home. Third, wear a mask when in public at all times, especially when indoors or when it is difficult to stay 6 feet away. The next video will tell you a bit more about masks. Video 2 (Control): Sugar is found in many different food items. Natural sugars are those that can be found in fruits, vegetables and dairy products like milk. Sugars like these that are natural are not really problematic because they are coming alongside lots of other vitamins and minerals. There are other sugars, though, that are processed and added to a food item. These are called additive sugars. A good rule of thumb is to eat foods with less than than 5g of sugar per serving. Avoid buying products where one of the first five products is a sugar. And it can be better to buy an unsweetened product like an unsweetened cereal or oatmeal and then add a teaspoon of sugar to it if you need the sweetness than to buy a heavily sweetened product, like a sugar cereal which can have several teaspoons of sugar per serving. Video 3 (Intervention): Hello, I am doctor [LAST NAME HERE] from [INSTITUTIONAL AFFILIATION HERE], and I will tell you a bit more about masks. Wearing a mask is a key way to prevent the spread of COVID-19. You are not just protecting yourself but also your grandma and your community, just in case you have COVID-19 but don't know it. Even if wearing a mask may sometimes put you in a difficult situation, it is important to protect you and the community from COVID 19 disease. As medical professionals, I am committed to delivering the best care I can to every patient. My goal is to make sure that you and everyone you love survives this COVID-19 pandemic. Thank you for listening to these messages. New fitness guidelines can be summed up as follows: just move and anything counts. Sneaking in a few minutes of physical activity throughout the day adds up in the long run. The guidelines are trying to make it easier for individuals to be fit and drop the rule that activity must be in 10 minute blocks of time. In a nutshell, activity has benefits even if it's for a short amount of time. Taking the stairs instead of the elevator, parking your car far away from the entrance to a store or walking your dog around the block can all help you be fit. The guidelines still call for at least 150 minutes a week of moderately intense aerobic exercise and two weekly sessions of muscle training activity, like lifting weights or yoga. This section reviews primary outcomes in more detail and describes secondary outcomes. The primary outcomes listed in ClinicalTrials.gov ID: NCT04502056 include "Knowledge Beliefs and Practices related to COVID-19" -we define knowledge gap outcomes in detail below which captures the knowledge and beliefs component. The seeking out of additional information and resources to protect oneself and one's family/community is the main behavioral outcome. Other primary outcomes are a safety gap outcome (for those who completed the follow up survey), the willingness to pay for a pair of reusable cloth masks and a charitable donation to a Black-targeted vs non targeted charity. The secondary outcomes in ClinicalTrials.gov ID: NCT04502056 and in the pre-analysis plan posted on https://www.socialscienceregistry.org/ included judgement of the federal and state policy responses, a charitable donation to COVID vs non COVID charity and a knowledge gap outcome of the follow up survey. They are shown in Supplementary Table 8a and 8b. Knowledge Gaps (primary outcome): We measure knowledge through 3 questions described below. Preventive practice: First, participants were asked to select three ways to prevent COVID-19 spread among a list that included staying six feet away from other people when outside, washing their hands when returning home and wearing a mask/facial covering when outside (See question "how_to_prevent" in the survey instrument). Each of these three practices that is not selected increases the count of knowledge gaps in preventive practice by 1, from = 0 up to = 3. Asymptomatic transmission: Second, participants were asked whether transmission by asymptomatic individuals is possible; those responding "no" were coded as having a knowledge gap for asymptomatic transmission ( = 1) while those responding "yes" were coded as having no knowledge gap for asymptomatic transmission ( = 0). When to wear a mask: Third, participants were asked to identify the two situations where it is most important to wear a mask in public places. Each selected practice different from "Indoors, at all times" or "Outdoors when it is impossible to stay six feet away from people" increases the count of knowledge gaps in mask practice by one, from = 0 = 2. Symptoms: Fourth, participants were asked about selecting exactly 4 common COVID-19 symptoms from a list. Each selected symptom that was not among cough, fever, difficulty breathing or a new loss of taste or smell increases the knowledge gap for symptoms by 1, from = 0 up to = 4. Knowledge Gaps Count: The knowledge gap count was defined as the sum of the three knowledge gaps ( = + + + ), taking any integer value between 0 and 10. This is our primary outcome. Information Seeking Behavior (primary outcome): Participants were asked to indicate interest in any number of links to COVID-related resources among a list of 5. There were told that they would subsequently obtain the selected links at the end of the study. We define the behavior index as the count of links selected. This could take any integer value between 0 and 5. The 5 links are: This outcome was measured a few days after the initial intervention, for a subsample that was eligible for follow up and could be tracked. Participants were asked about how often they engaged in four behaviors of interest (1. If they wore a mask indoors, 2. If they they wore a mask outdoors, 3. If they washed their hands, and 4. If they followed social distancing guidelines). A safety gap index was then calculated which has a value of 0 if they reported that they practiced the four behaviors of interest "always" up to 4 if they they report practicing none of them always. Knowledge Gaps follow up: It is the same outcome as the primary knowledge gap outcome, except that it is in the follow up survey. Willingness To Pay for Masks (primary outcome): Participants were asked how much a pair of reusable cloth masks is worth to them, i.e how much they would be willing to pay for them in a store. We define de WTP Mask index as the number of dollars selected by the respondent (between 0 and 40). Donation to GiveDirectly Project 100+: Participants were asked to choose how $1000 USD from the research team should be allocated between an organization that directly supports low-income families who need food assistance versus a fund that supports community organizations, with a focus on Black families who need food assistance and emergency support. Outcome is the amount allocated to the first one, an integer between 0 and 1000. Donation to BET COVID-19 Relief Fund (primary outcome): Participants were asked to choose how $1000 USD from the research team should be allocated between an organization that directly supports low-income families who need food assistance versus a fund that supports community organizations, with a focus on Black families who need food assistance and emergency support. Outcome is the amount allocated to the second one, an integer between 0 and 1000. Trust in Federal response: Participants were asked how well they think the federal government managed to balance opening the economy and limiting the health impacts of Covid-19. This outcome takes value 1 if the respondent answered "They managed the balance just right" Trust in Local response: Participants were asked how well they think the state government managed to balance opening the economy and limiting the health impacts of Covid-19. This outcome takes value 1 if the respondent answered "They managed the balance just right" We test for balance of baseline covariate distributions across intervention arms, first for the sample of individuals who were randomized in intervention or assigned to control and completed the baseline variables (Panel A of Supplement Table 1a) , then for the subsample of individuals who stayed in the study at least up to the knowledge outcome (Panel B), then for the subsample of individuals who completed the link questions (Panel C), and finally for the subsample of people who completed the knowledge questions in the follow up survey (Panel D). We use the function bal.tab() from the R package COBALT. We test for balance on marginal distribution of individual covariates, as well as for distribution of products of two covariates. We conduct these balance tests for all the intervention variations considered in our analysis: the group which receives intervention against the control group, the group which receives race concordant physician video against the group which does not, the group which receives an AMA statement acknowledging systemic racism against the group which receives an AMA placebo statement and the group which receives a message acknowledging increased incidence and mortality for Blacks against the group which received a standard message. For each comparison, we count the number of distributions that fail the balance test either by having a standardized mean difference greater than 0.1 or a Kolmogorov-Smirnov test p-value greater than 0.05, as recommended. Supplementary Table 1a report these counts for the full randomized sample, the sample who completed knowledge questions, the sample who completed links questions and the sample who completed knowledge questions in the follow up survey. We find no evidence of imbalance in all of these samples. To account for attrition in the follow up survey which can potentially confound our estimates, we conduct Hainmueller's entropy balancing in models we present in the main text, following (1). This data preprocessing method is designed to achieve covariate balance in observational studies with binary interventions. It calibrates individual weights to ensure that reweighted intervention and control groups satisfy a large set of balance conditions on first and second moments of the covariate distributions. The list of baseline covariates that are used to implement this reweighting procedure is the following: Stratum (which includes gender, an indicator for age below 44 years old, race, and self-reported Republican identification) -Household income above 60k -HS graduate - The 4 safety practices (mask in, mask out, wash hands, distance) -Prior belief "Blacks are more likely to die from COVID 19" Subsequently, these weights are used in the models that we present in the main text. For completeness, we also present the results from unweighted regressions in Supplement Tables 5 and 6 . Section E.1. Controlling for more baseline covariates using Double Post LASSO Here we describe how the Double Post Lasso method works for selecting additional control variables. We only use this method in additional analyses presented in Supplement Tables 3 and 4 . We do not include this approach in the primary analysis for simplicity and because it leads to very little precision gains in this application. We pre-specified to include baseline covariates chosen by a double-robust machine-learning algorithm (2) . This was used because it is a procedure delivering consistent estimates of intervention effects while improving efficiency by selecting covariates that are relevant to avoid omitted variable bias but exclude those that likely do not have such a threat. Technical details are in (2) . The procedure selects, in our case using LASSO, covariates that are correlated either with the outcome or with the intervention assignment. Notice both conditions can lead to omitted variable bias. So the LASSO procedure selects only relevant covariates that could have generated omitted variable bias (2). Let denote the outcome variable, the intervention variable and a vector of covariates. Here are the three steps of the Double Post Lasso selection method.  First, regress the intervention variable on the covariates using a Lasso regression. Let be the set of covariates which have a coefficient different from 0 in this regression.  Then, regress the outcome variable on the covariates using a Lasso regression. Let be the set of covariates which have a coefficient different from 0 in this regression.  Finally, fit the negative binomial regression models as described in the primary analysis where covariates in ∪ are included as regressors. For example, the intervention regression model now writes: Where is the vector of covariates from the set ∪ . Lasso regressions were implemented using the functions rlasso from the R package hdm v0.3.1. As detailed in Section D, to account for attrition which can potentially confound our estimates, we conduct Hainmueller's entropy balancing in models that we present in the main text, following (1) . For completeness, we also perform unweighted regressions, and we present the results in Supplement Tables 5 and 6 . Here we describe the regression models used for the analysis of secondary and additional outcomes presented in Section G and Supplement Tables 7 and 8. In all of our regressions, we adapted the model to the distribution of the outcome. For count variables (our primary outcomes), we fit a negative binomial model using glm.nb from the MASS package, as presented in the main text. For the binary outcome (interest in the DIY mask video), we used logistic regression with the glm function from the stats package. For every other variables (Donation, Perceived norms around masks and video ratings), we used OLS regression with the lm function from the stats package. We present the regression equations for logistic regression and OLS below. Negative binomial models are presented in the main text. In every regression, we control for strata. Below we show the regression equations. In all regression equations, refers to a vector of length 12 that indicates the participant's stratum with a 1 on its corresponding coordinate and zeros on other coordinates. Intervention Analysis: Sample: all intervention groups and control group. Participants who completed survey at least up to knowledge questions for our primary knowledge outcome. Participants who completed the entire survey for other outcomes. The coefficient of interest for Intervention is . Binary outcomes -logistic regression equation: where is the outcome variable, = ( = 1). = + +  Tailoring Intervention Analysis: Sample: Participants who were assigned to an intervention group (i.e. not assigned to the control group). Participants who completed the survey at least up to knowledge questions for our primary knowledge outcome. Participants who completed the entire survey for other outcomes. The coefficients of interest for Black Physician are . Binary outcomes -logistic regression equation: Other outcomes -OLS regression equation: All Black treatments Analysis: Sample: Participants who were assigned to an intervention group (i.e. not assigned to the control group). Participants who completed the survey at least up to knowledge questions for our primary knowledge outcome. Participants who completed the entire survey for other outcomes. In Supplement Table 12 , we estimate the following equations. Table 3 (with Panel = All participants). Incidence Rate Ratios (IRRs) for follow-up outcomes are calculated from estimates obtained by fitting a negative binomial regression model with units reweighted following Hainmueller's entropy-based reweighting to account for imbalances due to attrition (14). *This table presents number of observations and average incidence rates of knowledge gaps, information seeking behavior and safety gaps in the sample of participants who received intervention, split by whether they received a particular intervention or not. For instance, the first column shows the average incidence rate (and number of observations) for all participants that received the video message from a white physician, and the second column shows the average incidence rates (and number of observations) for all participants that received the video messages from a Black physician. 95% CI in parentheses. WTP = Willingness To Pay. Figure 2 shows the fraction of participants with a safety gap score of 0 (perfect answers), 1, 2, 3, or 4 (with confidence intervals in parentheses below each fraction) in the control group and in the intervention group. A Kolmogorov Smirnov test of the equality of the distribution has a value of 0.023486 (p=0.6593). This outcome was measured a few days after the initial intervention, for a subsample that was eligible for follow up and could be tracked. Participants were asked about how often they engaged in four behaviors of interest (1. If they wore a mask indoors, 2. If they wore a mask outdoors, 3. If they washed their hands, and 4. If they followed social distancing guidelines). Failure to answer "always" to each of these practices added one point to the safety gap count. The safety gap count is an integer that can have values from 0 (all safety practices) to 4 (no safety practice 0.78 0.64 18132 * This table presents incidence rates (or means/Odds) and incidence rate ratios (or coefficients/Odds ratios) from the intervention, disaggregated by education. Pvalues correspond to tests that the IRR/Odds ratios for low education (i.e. participants with less than high school education) and high education (i.e. participants with at least high school education) is equal to 1 (or 0 for OLS coefficients), and to a test that the IRR (or coefficients/Odds ratios) for participants with low education is different from the IRR (or coefficients/Odds ratios) for participants with high education levels, in order to test for heterogeneous effects. IRRs are estimated by fitting a negative binomial regression model (with units reweighted following Hainmueller's entropy-based reweighting to account for imbalances due to attrition for follow up outcomes) (1). Odds ratio are estimated by fitting a logistic regression. Coefficients for WTP Masks and Donation outcomes are obtained by fitting an OLS regression. CI = 95% confidence interval. WTP = Willingness To Pay. Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies Double/debiased machine learning for treatment and structural parameters *This table presents number of observations and average incidence rates for additional outcomes in the sample of participants who received intervention, split by whether they received a particular intervention or not. For instance, the first column shows the average incidence rate (and number of observations) for all participants that received the video message from a white physician, and the second column shows the average incidence rates (and number of observations) for all participants that received the video messages from a Black physician. 95% CI in parentheses.