key: cord-0709910-rdgfxj72
authors: Tan, Sophia T.; Park, Hailey J.; Rodríguez-Barraquer, Isabel; Rutherford, George W.; Bibbins-Domingo, Kirsten; Schechter, Robert; Lo, Nathan C.
title: COVID-19 Vaccination and Estimated Public Health Impact in California
date: 2022-04-22
journal: JAMA Netw Open
DOI: 10.1001/jamanetworkopen.2022.8526
sha: 209036c5d7435abab8db5a40dfa037230f25fcdb
doc_id: 709910
cord_uid: rdgfxj72

IMPORTANCE: Despite widespread vaccination against COVID-19 in the United States, there are limited empirical data quantifying their public health impact in the population. OBJECTIVE: To estimate the number of COVID-19 cases, hospitalizations, and deaths directly averted because of COVID-19 vaccination in California. DESIGN, SETTING, AND PARTICIPANTS: This modeling study used person-level data provided by the California Department of Public Health (CDPH) on COVID-19 cases, hospitalizations, and deaths as well as COVID-19 vaccine administration from January 1, 2020, to October 16, 2021. A statistical model was used to estimate the number of COVID-19 cases that would have occurred in the vaccine era (November 29, 2020, to October 16, 2021) in the absence of vaccination based on the ratio of the number of cases among the unvaccinated (aged <12 years) and vaccine-eligible groups (aged ≥12 years) before vaccine introduction. Vaccine-averted COVID-19 cases were estimated by finding the difference between the projected and observed number of COVID-19 cases. Averted COVID-19 hospitalizations and deaths were assessed by applying estimated hospitalization and case fatality risks to estimates of vaccine-averted COVID-19 cases. As a sensitivity analysis, a second independent model was developed to estimate the number of vaccine-averted COVID-19 outcomes by applying published data on vaccine effectiveness to data on COVID-19 vaccine administration and estimated risk of COVID-19 over time. EXPOSURE: COVID-19 vaccination. MAIN OUTCOMES AND MEASURES: Number of COVID-19 cases, hospitalizations, and deaths estimated to have been averted because of COVID-19 vaccination. RESULTS: There were 4 585 248 confirmed COVID-19 cases, 240 718 hospitalizations, and 70 406 deaths in California from January 1, 2020, to October 16, 2021, during which 27 164 680 vaccine-eligible individuals aged 12 years and older were reported to have received at least 1 dose of a COVID-19 vaccine in the vaccine era (79.5% of the eligible population). The primary model estimated that COVID-19 vaccination averted 1 523 500 (95% prediction interval [PI], 976 800-2 230 800) COVID-19 cases in California, corresponding to a 72% (95% PI, 53%-91%) relative reduction in cases because of vaccination. COVID-19 vaccination was estimated to have averted 72 930 (95% PI, 53 250-99 160) hospitalizations and 19 430 (95% PI, 14 840-26 230) deaths during the study period. The alternative model identified comparable findings. CONCLUSIONS AND RELEVANCE: This study provides evidence of the public health benefit of COVID-19 vaccination in the United States and further supports the urgency for continued vaccination.

In this appendix, we provide further methodologic detail on the model structure and statistical analysis.

Calculation of relative reduction of COVID-19 cases We estimated the relative reduction of COVID-19 cases in the entire vaccine-eligible population (≥12 years) and each age group (12-17 years, 18-49 years, 50-64 years, and ≥65 years) after the start of Phase 1A of vaccination (November 29, 2020), adjusting for vaccine coverage in the relevant population. The formula for percentage reduction in COVID-19 cases over a fixed period of time is as follows:

In sensitivity analysis, we estimated alternative formulations of the relative reduction: (1) accounting for age-specific eligibility over time; and (2) not adjusting by vaccine coverage (see Sensitivity analyses).

We defined the lower bound for the number of weekly averted COVID-19 cases as zero based on bioplausibility.

Model of natural immunity We assumed that natural infection provided perfect immunity without waning though we relaxed this assumption in a sensitivity analysis. We assumed complete reporting of COVID-19 cases. We estimated total infections in the unvaccinated and each vaccine-eligible age group (<12 years, 12-17 years, 18-49 years, 50-64 years, ≥65 years) using literature estimates of the subclinical proportion by age (<19 years, 19-59 years, ≥60 years) 1 . We used the reported means and 95% confidence intervals of each age-specific subclinical fraction of infection to fit optimal beta distributions. The mean and the fitted shape parameters of each distribution are shown in Table A1 . 

We modeled vaccine effectiveness (against clinical disease) and waning immunity on a personlevel based on vaccine (BNT162b2, mRNA-1273, Ad26.COV2.S) and the number of doses received. We assumed six possible vaccination scenarios: 1) BNT162b2 single dose; 2) BNT162b2 two doses; 3) mRNA-1273 single dose; 4) mRNA-1273 two doses; 5) Ad26.COV2.S single dose; and 6) unvaccinated. We did not include boosters given limited use over the study period.

We used published literature to estimate the vaccine effectiveness in each scenario over time, assuming instantaneous onset of protection and waning immunity at various time points 2-6 . We fit beta distributions using the published mean and 95% confidence intervals of each estimate of vaccine effectiveness (see Table A2 ). We made the simplifying assumption that all individuals who received two doses of the BNT162b2 vaccine received their second dose three weeks after their first dose and all individuals that received two doses of the mRNA-1273 vaccine received their second dose four weeks after receiving their first dose based on published literature 7 . We did not account for potential differences in vaccine effectiveness by age, which is broadly supported by literature 6, 8, 9 . We accounted for possible changes in vaccine effectiveness against the highly infectious Delta variant of SARS-CoV-2 as a sensitivity analysis (see Sensitivity analyses) but did not account for variant specific effectiveness in the main analysis. Average vaccine effectiveness and waning over time is shown in Figure A1 , and the distributions of vaccine effectiveness are shown in Table A2 . We used publicly available COVID-19 vaccination data 10 to estimate the weekly number of newly vaccinated individuals in each of the six scenarios and age groups (12-17 years, 18-49 years, 50-64 years, ≥65 years). These age groups were based on vaccine prioritization age groupings. Date of receipt of first and second doses of the BNT162b2 and mRNA-1273 vaccines was not available in our data. We therefore calculated the mean fraction of individuals that received BNT162b2 or mRNA-1273 vaccines in each age group to estimate weekly BNT162b2 and mRNA-1273 vaccinations. We additionally used published literature to estimate the proportion of individuals who received only a single dose of the BNT162b2 or mRNA-1273 vaccines 7 .

We combined our estimates of the number of individuals newly vaccinated each week by vaccine type and number of doses received and corresponding vaccine effectiveness over time to estimate the fraction of the population with immunity due to vaccination. Since we assumed that natural infection provided perfect immunity, we first calculated the number of newly vaccinated individuals not previously infected with SARS-CoV-2 each week before estimating the number of protected individuals over time due to vaccination. We assumed previously infected and uninfected individuals were equally likely to receive any COVID-19 vaccine, following the observed weekly distribution of vaccines by vaccine type. The formula for calculating the number of new vaccinations in previously uninfected individuals in an age group a at week t is as follows:

We used Monte Carlo simulation to capture uncertainty for the analysis in the alternative model, with a focus on accounting for uncertainty in vaccine effectiveness and estimates of subclinical infection. We ran 1000 simulations using randomly sampled values of parameters from fitted parameter distributions (Table A1 and A2). We reported the mean and 95% uncertainty intervals (95% UI) of study outcomes.

To generate random samples of our parameters for each simulation, we independently sampled from the distributions of sub-clinical fractions in three age groups: <19 years, 19-59 years, and ≥60 years. We sampled independently from the standard uniform distribution for three vaccines (BNT162b2, mRNA-1273, and Ad26.COV2.S) and used inversion sampling to generate samples of vaccine effectiveness to account for changes in effectiveness over time.

Estimation of averted COVID-19 hospitalizations and deaths: Additional methodology We estimated monthly risks of hospitalization and death in each age group of the population (<12 years, 12-17 years, 18-49 years, 50-64 years, ≥65 years) by finding the proportion of cases that resulted in hospitalization or death each month using CDPH data.

Due to lag in reporting of severe COVID-19 outcomes, we used the age-specific monthly risk of hospitalization and death in August 2021 to predict averted hospitalizations and deaths in September and October 2021 (eFigure 1). We defined the lower bound for the number of weekly averted COVID-19 hospitalizations and deaths as zero based on bioplausibility.

We also used values from literatures for risk of hospitalization and death from COVID-19 in sensitivity analysis.

Prediction and uncertainty intervals Prediction intervals from the primary modeling approach reflect both uncertainty in the estimated model parameters and variation expected for the outcome, while the uncertainty intervals from the alternative modeling approach reflect the uncertainty in the parameters inputted into the alternative model. These represent different measures of statistical variability in estimation.

Age-specific vaccine eligibility In both modeling approaches, we performed a sensitivity analysis to account for age-specific differences in COVID-19 vaccine eligibility over time among the four vaccine-eligible age groups (12-17 years, 18-49 years, 50-64 years, ≥65 years).

COVID-19 vaccines became available for the general population ≥16 years and 12-15 years in mid-April 2021 and mid-May 2021 respectively 11 . We assumed vaccination in the population 12-17 years began on April 11, 2021.

Vaccination in adults 18-49 years and 50-64 years began before vaccines were widely available in those populations due to occupational risk. Healthcare and other frontline workers became eligible for COVID-19 vaccines in Phase 1A of vaccination and essential workers became eligible for vaccines in Phase 1B of vaccination 11, 12 . For this analysis, we assumed widespread vaccination in the populations 18-49 years and 50-64 years began with Phase 1B of vaccination and used February 14, 2021 to mark vaccine eligibility [13] [14] [15] [16] .

We used January 10, 2021 as the start of vaccination among adults ≥65 years, since they were eligible for COVID-19 vaccines beginning mid-January 2021 17 .

We performed the main analysis in each age group after each age group became eligible for vaccines, reporting both unadjusted and adjusted relative reduction of outcomes. Dates of vaccine-eligibility by age group used in this analysis are shown in Table A3 . Table A4 . We did not perform this sensitivity analysis in the primary model since person-level vaccination was not explicitly included. We conducted a separate sensitivity analysis that relaxed the assumption of perfect immunity from infection. We assumed that SARS-CoV-2 infection was 86% effective against reinfection within 1 year of primary infection with waning after 1 year based on recent published literature 22, 23 . We included these estimates of effectiveness of natural infection against reinfection as additional parameters to sample from in the Monte Carlo simulation. The fitted beta distributions of these parameters are shown in Table A5 . We additionally assumed that all COVID-19 vaccines were 90% effective against reinfection after previous natural infection, which is supported by literature 22 . 

We assessed hospitalization and death outcomes of the main analyses when using literature estimates of the risk of hospitalization and death in cases that were not fully vaccinated 25 rather than estimates from CDPH data. A comparison of the hospitalization and death risk used in the main analyses and literature estimates are shown in Table A6 . The alternative model we developed is applicable to other COVID-19 outcomes. As an additional sensitivity analysis, we adapted the alternative model to predict hospitalizations and deaths that would have occurred in the absence of vaccination. We estimated the incidence of hospitalization and deaths instead of incidence of cases, incorporating literature estimates of vaccine effectiveness against hospitalizations and death to estimate the susceptibility profile of the population. The distributions of vaccine effectiveness against hospitalization and death are in Table A7 . 

We assumed vaccine eligibility began for 12-17 years on

** Relative reduction in outcomes were adjusted for the mean vaccine coverage in the population during the vaccine era, while the unadjusted estimate did not account for vaccine coverage COVID-19 hospitalization

36) 53 (47, 59) Relative reduction in outcome (%) (95% UI) * Outcome Unadjusted Adjusted COVID-19 hospitalization ≥18

Asymptomatic SARS-CoV-2 infection: A systematic review and meta-analysis

Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine

Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine

Safety and Efficacy of Single-Dose Ad26.COV2.S Vaccine against Covid-19

Comparative Effectiveness of Moderna

Interim Estimates of Vaccine Effectiveness of Pfizer-BioNTech and Moderna COVID-19 Vaccines Among Health Care Personnel -33

COVID-19 Vaccine Second-Dose Completion and Interval Between First and Second Doses Among Vaccinated Persons -United States

Effectiveness of Pfizer-BioNTech mRNA Vaccination Against COVID-19

Hospitalization Among Persons Aged 12-18 Years -United States

Effectiveness of COVID-19 Vaccines in Preventing Hospitalization Among Adults Aged ≥65 Years -COVID-NET, 13 States

COVID-19 Vaccine Progress Dashboard Data -COVID-19 Vaccines Administered By Demographics -California Health and Human Services Open Data Portal

Updated COVID-19 Vaccine Eligibility Guidelines

The Advisory Committee on Immunization Practices' Updated Interim Recommendation for Allocation of COVID-19 Vaccine -United States

San Francisco Moves to Phase 1B of COVID-19 Vaccinations, Expands Eligibility to Educators, Child Care, Emergency Services, Food and Agriculture Workers | Office of the Mayor

Alameda County Gets a Mega Vaccination Site through State-Federal Partnership

Contra Costa Extends COVID-19 Vaccine Eligibility to Essential Workers :: Press Releases :: Contra Costa Health Services

Seniors 65+ Now Eligible to Receive COVID-19 Vaccine to Effectively and Efficiently Increase Vaccine Distribution, Reduce Hospitalizations and Save Lives

Effectiveness of mRNA-1273 against delta, mu, and other emerging variants of SARS-CoV-2: test negative case-control study

Effectiveness of the BNT162b2 Covid-19 Vaccine against the B.1.1.7 and B.1.351 Variants

Effectiveness of mRNA BNT162b2 COVID-19 vaccine up to 6 months in a large integrated health system in the USA: a retrospective cohort study. The Lancet

Interim Estimates of COVID-19 Vaccine Effectiveness Against COVID-19-Associated Emergency Department or Urgent Care Clinic Encounters and Hospitalizations Among Adults During SARS-CoV-2 B.1.617.2 (Delta) Variant Predominance -Nine States

Protection against SARS-CoV-2 after Covid-19 Vaccination and Previous Infection

Efficacy of Natural Immunity against SARS-CoV-2 Reinfection with the Beta Variant

Duration of Protection against Mild and Severe Disease by Covid-19 Vaccines

Monitoring Incidence of COVID-19 Cases, Hospitalizations, and Deaths, by Vaccination Status -13

Effectiveness of Covid-19 Vaccines over a 9-Month Period in North Carolina

Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: Updated reporting guidance for health economic evaluations

We compare the results estimating averted COVID-19 cases (A, B), hospitalizations (C, D), and deaths (E, F) in the population ≥65 years from the primary model (left) and the alternative model (right). In both panels, we plot the vaccine coverage of at least 1 dose of a COVID-19 vaccine over the vaccine era (red). We plot the observed outcome over time in green and the predicted outcome in the absence of vaccination from each model in blue. The difference between the predicted outcome in absence of vaccination and the observed outcome represents the averted outcome due to COVID-19 vaccination. The dashed line represents the introduction of the Delta variant in California in June 2021 (black).