key: cord-1038405-tn6suy9k
authors: Mauskopf, Josephine; Masaquel, Catherine; Huang, Liping
title: Evaluating Vaccination Programs That Prevent Diseases with Potentially Catastrophic Health Outcomes: How Can We Capture the Value of Risk Reduction?
date: 2020-10-13
journal: Value Health
DOI: 10.1016/j.jval.2020.06.018
sha: c1ab3d4560843262b4d3f2f3b5a722b26159db20
doc_id: 1038405
cord_uid: tn6suy9k

In the last 5 years, guidelines have been developed for performing cost-effectiveness analyses (CEAs) for the economic evaluation of vaccination programs against infectious diseases. However, these cost-effectiveness guidelines do not provide specific guidance for including the value of reducing the risk of rare but potentially catastrophic health outcomes, such as mortality or long-term sequelae. Alternative economic evaluation methods, including extended CEA, the impact inventory, cost-benefit analyses, willingness to pay or the value of a statistical life, to capture the value of this risk reduction could provide more complete estimates of the value of vaccination programs for diseases with potentially catastrophic health and nonhealth outcomes. In this commentary, using invasive meningococcal disease as an example, we describe these alternative approaches along with examples to illustrate how the benefits of vaccination in reducing risk of catastrophic health outcomes can be valued. These benefits are not usually captured in CEAs that only include population benefits estimated as the quality-adjusted life-years gained and reduced costs from avoided cases.

In the last 5 years, 3 publications have presented guidelines for the economic evaluation of vaccination programs against infectious diseases using cost-effectiveness analyses (CEAs). 1-3 All 3 guidelines provide consistent recommendations for best practice in performing CEAs of vaccines based on vaccination program costs and estimates of the impact of the program on disease cases and the associated health, costs, and quality-of-life outcomes. The World Health Organization guidelines provide flow diagrams to assist with the epidemic model choice. 1 The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) guidelines describe 2 alternative approaches to CEA when performing economic evaluations of vaccination programs: constrained optimization across all disease-management options and fiscal health modeling to estimate the value of the program from the government perspective. 2 The ISPOR guidelines define 3 types of input data to be used in all economic evaluations of vaccination programs: "epidemiologic data and competing causes of death in the population of interest; vaccination trial results, such as efficacy and protection duration; and economic and health outcome inputs such as prevention program-related and disease-related resource use and costs and disease-related QALYs [quality-adjusted lifeyears] or DALYs [disability-adjusted life-years]." 2 Ultsch et al 3 developed a consensus framework based on a systematic literature review and feedback from experts in health economic evaluation and immunization decision-making in Europe, and concluded that a health economic evaluation should be done in a consistent way across all interventions, including treatments and vaccines. All 3 guidelines recognize that disease-related costs may include nonhealth-related costs such as productivity losses and reduction in educational attainment. [1] [2] [3] However, current cost-effectiveness guidelines for vaccination programs do not provide specific guidance for economic evaluation when the disease being prevented is rare but may have catastrophic health outcomes such as death or long-term serious sequelae, including deafness or neurological impairment. For many common vaccine-preventable diseases, the resource impact of preventing cases of the disease will be much higher than the resource impact of preventing a disease that is rare. However, a rare disease might have a higher per-case rate of catastrophic outcomes such as severe morbidity and mortality. 4 Although an economic evaluation of a vaccination program may follow the current cost-effectiveness guidelines, the analysis results may underestimate to society the value of reducing the risk of the disease for all persons vaccinated as well as for the unvaccinated population. This benefit could be of particular importance for diseases where there is a risk of catastrophic health outcomes such as death or long-term sequelae, as noted previously by Drummond et al. 5 Alternative economic evaluation methods can be used, including estimates of willingness to pay (WTP) or value of a statistical life (VSL) to estimate the value of a reduction in the risk of mortality. 6 Including estimates using these methods might provide more complete estimates of the value of vaccination programs for diseases with potentially catastrophic outcomes such as invasive meningococcal disease (IMD), measles, pneumonia, pertussis, varicella, rubella, and human papillomavirus (HPV) infection. These methods might also provide more complete estimates of the value of new disease treatments that reduce the risk of catastrophic disease outcomes, such as a treatment for COVID-19 that reduced the risk of mortality.

In this commentary, we focus on IMD because the traditional CEA often underestimates the value of preventing this disease with potentially catastrophic outcomes. For example, compared with varicella, the incidence of IMD is much lower and therefore the overall health and nonhealth resource use due to varicella is much higher than that for IMD. With the traditional CEA approach, the resources saved from avoided cases may be lower for IMD than for varicella. However, the case fatality rate and longterm morbidity rates are much higher for IMD compared with varicella. 4 Thus, although the number needed to vaccinate to prevent a case would be much lower for varicella than for IMD, the number needed to vaccinate to prevent one death from varicella would likely be closer to that for IMD. We first present a brief description of IMD followed by brief descriptions of methods to estimate the value of new vaccination programs including CEA, extended CEA, and cost-consequence listings, as well as costbenefit analysis (CBA) using WTP or VSL methods to estimate the value of the reduced risk of catastrophic health outcomes. We then provide some examples of how these methods have been used to estimate the value of vaccination programs for IMD and other infectious diseases with potentially catastrophic health outcomes. We end by discussing how these alternative economic valuation methods might be used to capture the full value of investments in vaccines to prevent infectious diseases and to inform vaccination program recommendations.

Invasive meningococcal disease is a serious, life-threatening infection caused by the bacterium Neisseria meningitidis, with early stages that are nonspecific, resembling viral infections or pneumonia. 7 Even with appropriate treatment, 5% to 10% of patients can die within 24 to 48 hours after the onset of symptoms, and approximately 8% to 20% of survivors are left with serious sequelae, such as deafness and neurological impairments. 7, 8 The majority of cases of IMD are caused by 6 serogroups (A, B, C, W, X, and Y). The epidemiology of IMD naturally fluctuates over time, both in magnitude (including endemic, epidemic, and hyperepidemic periods) and in the most prominent disease-causing serogroups (including B, C, W, and Y) as well as in the age groups most affected. 9,10 Invasive meningococcal disease is rare and has been decreasing over the past 2 decades, which is partly attributable to implementation of meningococcal vaccination programs. 11 Annual incidence in the United States decreased from 1.15 cases per 100 000 in 1996 to 0.12 cases per 100 000 in 2015. 12 In European member states, 13 the annual incidence decreased from 1.13 cases per 100 000 in 2004 to 0.55 cases per 100 000 in 2014.

Cost-effectiveness analysis estimates an incremental costeffectiveness ratio (ICER) that is computed as the ratio of the difference in vaccination program plus disease treatment costs between 2 vaccination programs (or between a new vaccination program and no program for the same disease) and the difference in the health outcomes expected. In published CEAs of IMD and other disease vaccination programs, the ICERs are either cost per quality-adjusted life-year (QALY) gained or cost per life-year gained. The QALYs or life-years gained are usually estimated based on estimated avoided IMD cases, sequelae, and deaths resulting from the implementation of a vaccination program. In the United Kingdom, the threshold value for an ICER to determine cost-effectiveness is between £20 000 and £30 000. The corresponding US threshold value used for an ICER is now between $100 000 and $150 000. Other countries have different threshold values. The threshold values are typically applied to all conditions and all types of healthcare interventions and are not used to consider differences in disease characteristics. Estimated ICERs for meningococcal vaccination programs for different target populations have been higher or lower than these threshold values, depending primarily on the estimates of incidence in the target population and the price and number of doses required.

A framework that went beyond cost-effectiveness for evaluating a new vaccination program was proposed by Erikson et al 14 in 2005. This framework included disease burden, vaccine efficacy and safety, and cost-effectiveness as well as implementation, feasibility, and acceptability of the program. An article by Crowcroft et al 15 used this framework to assess the value of meningococcal group B vaccine (Bexsero; Novartis, Basel, Switzerland) for prevention of IMD serogroup B. In their assessment, a rating of low disease burden was assigned based on the low incidence of the disease. 15 However, in this framework for the assessment of disease burden it was not clear how to balance disease incidence rates (high or low) with high case fatality and long-term sequelae rates (high or low) when determining the rating of disease burden. 15 Bloom et al 16 emphasize the importance of further research into the impacts of vaccination beyond that on the healthcare of the individual vaccinated, including educational attainment, cognitive development, labor productivity, income, savings, investment, and fertility. They suggest that currently the value of vaccines is underestimated because these additional factors for the person vaccinated are not necessarily included in the analyses. 16 A similar listing of a broad range of benefits attributable to a vaccination program were enunciated by Lakdawalla et al 17 as a value "flower" in an ISPOR value frameworks special task force report and was presented in Appendix B of the ISPOR economic evaluation of vaccination programs task force report. 2 In addition, the second Preventive Task Force 18 also suggested that a societal perspective for all prevention programs should include an impact inventory, listing all the potential benefits of a vaccination program to the extent that credible estimates are available.

Additional outcomes such as lifetime productivity losses and household financial risk could also be included in an "extended CEA" 19 for an illness such as IMD that leaves some of the survivors with long-term serious sequelae, such as deafness and neurological impairment. Extended CEAs have been published for vaccination programs such as HPV and pneumonia. 20, 21 For HPV vaccination, the extended CEA included a broader set of cancers associated with HPV infection beyond cervical cancer, including anal and penis cancer, and for pneumonia vaccination, the extended CEA included household expenditures, financial risk protection for the family, and population distributional effects.

A reanalysis of the cost-effectiveness of Bexsero for IMD serogroup B in the United Kingdom extended the original traditional CEA 22 to include additional outcomes attributable to the vaccination program such as quality-of-life gains for parents of protected infants and a share of the avoided IMD-related litigation costs to the National Health Service. 23 Although the results from the original CEA indicated that Bexsero would not be cost-effective at any price, the second analysis gave more favorable results and the Joint Committee on Vaccination and Immunisation (JCVI) added Bexsero to the infant vaccination schedule.

Cost-benefit analysis (also referred to as benefit-cost analysis) is another method that can be used in the evaluation of alternative vaccination programs. Cost-benefit analysis has been used as an alternative to CEA for evaluating environmental regulations as well as healthcare interventions. 16 In this analysis, health and nonhealth benefits from the intervention are converted to monetized benefits. Health benefits are generally estimated as QALYs, life-years gained, or reduced risk of mortality and morbidity. Nonhealth benefits may include financial risk protection, future learning, and productivity gains, which can be substantial, particularly for vaccines preventing disease with potentially catastrophic health outcomes. The result may be presented as a ratio of benefit to cost and return on investment or as a net monetary benefit, estimated as the difference between the health and nonhealth benefits and the costs of the vaccination program. However, CBA is comparatively more complex, as every benefit needs to be quantified and requires explicit consideration of the monetary value.

For vaccines that reduce the risk of disease with potentially catastrophic health outcomes, the net monetized benefits could be estimated as the difference between the costs of the vaccination program and the monetized value of the reduced number of cases measured as reduced healthcare resources for treatment and QALYs gained from the avoided cases. To monetize the benefits from the QALY gains, the regional threshold values used for an assessment of the ICERs from a CEA may be applied to the QALY gains to convert estimates of cost per QALY gained into estimates of the monetary health benefits of the vaccination program. Alternatively, the net monetized benefits could be estimated as the difference between the costs of the vaccination program and the monetized value of the reduced risk of experiencing the catastrophic health outcomes associated with cases of the disease. The benefits of a reduced risk can be monetized by using an estimate of the VSL, which is defined as the value of reducing the risk of mortality and morbidity. The VSL has been estimated in several different ways: on the basis of revealed preference through the observed willingness to accept compensation for a higher risk (eg, as observed higher wages for riskier jobs) 24 or WTP for a lower risk (eg, observed higher prices for houses that are not near a hazardous waste site), 25 direct elicitation of WTP to lower the risk or mortality or other serious outcomes from disease cases (eg, for a vaccine for pneumococcal disease), 26 and a stated-preference survey (eg, for choices between alternative options for reducing the risk of dying over the next 10 years but with different costs). 27 The VSL is a crucial component of the benefit-cost analyses, which is often used by government agencies to decide whether a proposed regulation or a health program is worth the cost. The VSL has been used by various US federal agencies to decide whether to implement a regulation. The VSL is also used in many other countries as part of CBAs. 28 The US Environmental Protection Agency has used a value of approximately $9 million as the VSL in 2015 US dollars. 29 However, estimates of VSL derived by using willingness to accept or WTP methods may give different values of VSL, 30 and meta-analyses of published results from studies selected using different criteria and different health risk data sources may result in VSL estimates that range from $3.5 million to $11.4 million. 31 Park et al 32 compared the costs and benefits of an HPV vaccination program in the United Kingdom using different methods to monetize the health outcomes. They first computed the net monetary benefits as well as the ratio of total monetized benefits to total costs (benefit-cost ratio) for HPV vaccination using the results of a CEA that estimated QALYs gained from avoided cases of cervical cancer attributable to HPV vaccination by applying a value of £23 000 per QALY gained. 33 They next computed the net monetary benefits and benefit-cost ratio by assigning a value of £7.2 million (estimated as the VSL based on a meta-regression analysis of published data and purchasing power parity conversion to UK currency 25 ) to their estimates of the reduced risk of dying from cervical cancer attributable to the HPV vaccination programs. Finally, the researchers used the 2 estimates of monetary benefits and costs to compute the maximum vaccine price below which the benefit-cost ratio was .1 (meaning that the benefits outweigh the costs). They found that when computing the monetized benefits using the reduced risk of dying from cervical cancer and a VSL of £7.2 million, the maximum vaccine price was £1417 compared with that when using the results of the traditional CEA and a QALY threshold of £23 000, which resulted in a maximum vaccine price of £262. Their analysis clearly showed that an ICER based on a UK threshold value of £23 000 as used by the health technology assessment agencies might not fully capture the value of the health outcomes attributable to a vaccination program when one of the possible disease outcomes was a reduction in mortality from a rare but catastrophic event.

Direct elicitation of willingness-to-pay estimates for a vaccination program for a specific disease is another approach that has been used to capture the value of implementing a vaccination program that prevents rare but catastrophic events, and it also can be used for estimating VSL. A study conducted by Prosser et al 26 presented to US parents and other adults a description of meningitis and associated death and long-term sequelae as a rare complication of pneumococcal pneumonia in infants and young children. The study found that US parents and other adults were willing to pay $500 to reduce the risk of pneumococcal meningitis in young children from 21 per 100 000 to 6 per 100 000, 26 which is equivalent to~$3.3 million per case of pneumococcal meningitis avoided. A second study in France and Germany by Bishai et al 34 estimated parents' WTP for a meningococcal vaccine covering one or more serogroups for teenage children. Parents were given a description of IMD that included annual incidence by serogroups and frequency and severity of complications such as amputation, brain damage, and death. The results indicated that most parents in France and Germany were willing to pay V50 out of pocket for a meningococcal vaccine "deemed not 'cost-beneficial' by a team of economists." 34 A stated-preference, discrete-choice experiment in Australia by Wang et al 35 estimated that adolescents and their families were willing to pay AU $394.28 more for a vaccine that prevented lifethreatening illness than for a vaccine that prevented mild to moderate illness.

The examples above provide quantitative support for the point made by Drummond et al 5 that there is a value to the risk reduction that occurs upon vaccination for persons vaccinated and their families and for the population in which they reside. This value is not included in CEAs that include as benefits only the QALY gains and reduced treatment costs from avoided cases. Thus, although published CEAs for IMD (for example, de Wals and Zhou et al 36 23 ) include QALYs gained and costs avoided from the reduced IMD cases and associated reduction in mortality and severe complications attributable to the vaccination program, they do not necessarily estimate the full value to children, adolescents, parents, and society of reducing the risk of mortality and morbidity from IMD, which could be estimated using CBA and VSL methods and WTP methods. In addition to including the value of the risk reduction from a vaccination program, both CEAs and CBAs need to include the acquisition costs of the vaccination program both initially and after patent protections have expired, the costs of implementing and running the program, other reduced or increased diseaserelated costs associated with vaccination, and other health and nonhealth benefits either monetized or included as QALYs or in an impact listing.

Although CEAs of vaccination programs for IMD have not explicitly included estimates of the value of the reduced risk of mortality or long-term sequelae explicitly, these factors might have been implicitly included in decisions about vaccination programs for IMD. When outbreaks or even a single case of IMD occurs, there is often pressure from parents, physicians, and the media with support from the public health authorities to implement a mass or routine immunization program without consideration of the cost-effectiveness of such a program because of the catastrophic nature of IMD. For example, a CEA in Quebec revealed that switching from the serogroup C meningococcal vaccine to the serogroup ACWY meningococcal vaccine for routine vaccination in adolescents might not be cost-effective because of the low incidence of IMD. 36 Yet because the vaccine would reduce cases of IMD, all other provinces in Canada have made this switch. In another example, widespread protests followed an initial decision in the United Kingdom not to provide vaccination against serogroup B meningococcal (MenB) disease to infants based on an initial traditional CEA. This protest resulted in a reassessment based on an extended CEA, and thus led to a recommendation to implement a MenB program for infants. 37 The extended CEA that supported the final decision by the JCVI included an extended set of outcomes including QALY losses for families of those with IMD and litigation costs for the National Health Service, 23 as well as the use of lower discount rates. The decision by JCVI to recommend use of a MenB vaccine in infants despite unfavorable initial ICERs indicates that they recognized that catastrophic outcomes, even if rare, do have added value beyond the patient's QALYs lost from avoided cases.

As we have shown, these alternative estimation methods, such as benefit-cost analysis using the VSL or WTP estimation, can be applied to the evaluation of vaccination programs for infectious diseases with rare but potentially catastrophic health events. In addition, they can be applied to other healthcare interventions that prevent rare but potentially catastrophic health events. Such events in otherwise healthy children or young adults include lifethreatening allergic reactions in school settings in children with either known or unknown allergies 38 and risk of sudden death from cardiac events in young athletes (,35 years of age) without known cardiovascular disease and at low risk. 39 Interventions to reduce the risk of these events, such as epinephrine injectors at schools and preparticipation in cardiovascular screening for individuals participating in athletic programs, are available, but as with vaccination programs, these interventions may be undervalued by healthcare decision-makers following current guidelines for CEA because of the low risk of the catastrophic event. The article by Fishbein 39 cited data from a northern Italian preparticipation screening program for competitive athletes, which has been ongoing for 26 years and which has reduced sudden cardiac deaths in young athletes by 89%. 40 Corrado et al 40 estimated that it cost $40 to $45 for each screen, and Fishbein indicated that given the observed number of sudden cardiac deaths, with this program you would need to screen 200 000 people to avoid one death. Based on this information, the program resulted in expenditures of approximately $8 million to $9 million per death avoided. This expenditure per death avoided is consistent with estimates of the VSL that might be used in a CBA but would likely result in a cost per QALY gained for the screening program that is higher than typical CEA threshold values.

O'Mahony and Paulden, 41 in a commentary expressing concern about the "bending of CEA guidelines," questioned the fairness of the UK JCVI's use of lower discount rates than the generally accepted values for CEAs to justify their recommendation for the immunization of adolescent boys with the HPV vaccine. O'Mahony and Paulden 41 also expressed concern about the JCVI's use of an "arbitrary" QALY adjustment factor of 3 to justify a recommendation for MenB vaccination in infants. This view suggests it might be time to modify the economic evaluation guidelines for all healthcare interventions, as suggested by O'Mahony and Paulden, 41 to explicitly include quantitative estimates of the value of reducing the risk of catastrophic disease outcomes as well as the additional outcomes proposed recently. [16] [17] [18] [19] With the ongoing efforts to develop a vaccine for the COVID-19 pandemic, the value of vaccines that reduce the risk of catastrophic outcomes both health and economic is now more clearly understood but would not necessarily be captured in a CEA following current guidelines, which might capture only the benefits of the QALYs gained from avoiding symptomatic cases. As we have shown, the alternative methods described in this commentary have been used to value the prevention of rare but catastrophic health outcomes for vaccination programs. They have also been used to value other public services that prevent catastrophic health outcomes (eg, road safety, workplace safety, air and water pollution, nuclear power plants) and could also be used by vaccination decision-makers to evaluate alternative vaccination programs. Although Verguet et al 19 have developed methods to estimate household financial risk, new methods are currently needed to estimate the risks to a regional or international economy to capture the value of a vaccination program or effective treatment for a pandemic. Author Contributions: All authors contributed to formulate concepts, identify important intellectual content, and data interpretation. JM drafted the manuscript. All authors reviewed and edited the drafts of the manuscript and have given final approval of the version to be published.

Conflict of Interest Disclosures: LH is an employee of Pfizer who have developed vaccines for IMD. JM and CM are employees of RTI Health Solutions who received research funding from Pfizer Inc.

Funding Support: The study was supported by a contract from Pfizer to RTI Health Solutions.

Role of the Funder/Sponsor: One of the commentary authors was an employee of the Funder and so had a role in the design and conduct of the study, and analysis and review of the information included and approval of the manuscript and decision to submit the manuscript for publication.

World Health Organization. WHO Guide for Standardization of Economic Evaluations of Immunization Programmes

Economic analysis of vaccination programs: an ISPOR good practices for outcomes research task force report

Methods for health economic evaluation of vaccines and immunization decision frameworks: a consensus framework from a European vaccine economics community

The role of health economic analyses in vaccine decision making

Do we fully understand the economic value of vaccines? Vaccine

The social value of mortality risk reduction: VSL versus the social welfare function approach

Neisseria meningitidis: epidemiology, treatment and prevention in adolescents

Weekly epidemiological record: meningococcal vaccines -WHO position paper

The Global Meningococcal Initiative: global epidemiology, the impact of vaccines on meningococcal disease and the importance of herd protection

Meningococcal serogroups and surveillance: a systematic review and survey

The global evolution of meningococcal epidemiology following the introduction of meningococcal vaccines

Guidance for the evaluation of public health management of suspected outbreaks of meningococcal disease. Advisory Committee on Immunization Practice. Version 2.0

The epidemiology of invasive meningococcal disease in EU/EEA countries

An analytical framework for immunization programs in Canada

Do we need a new approach for making vaccine recommendations?

Moving beyond traditional valuation of vaccination: needs and opportunities

Defining elements of value in health care-a health economics approach: an ISPOR Special Task Force

Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: Second Panel on Cost-Effectiveness in Health and Medicine

Extended cost-effectiveness analysis for health policy assessment: a tutorial

Economic evaluation of vaccination: capturing the full benefits, with an application to human papillomavirus

Health gains and financial protection from pneumococcal vaccination and pneumonia treatment in Ethiopia: results from an extended cost-effectiveness analysis

Introducing vaccination against serogroup B meningococcal disease: an economic and mathematical modelling study of potential impact

Re-evaluating cost effectiveness of universal meningitis vaccination (Bexsero) in England: modelling study

The value of risks to life and health

The market value of reducing cancer risk: hedonic housing prices with changing information

Preferences and willingness to pay for health states prevented by pneumococcal conjugate vaccine

Age, health and the willingness to pay for mortality risk reductions: a contingent valuation survey of Ontario residents

Valuing vaccines using value of statistical life measures

Averting catastrophes that kill. NBER Working Papers 23346

Reference-dependence effects in benefit assessment: beyond the WTA-WTP dichotomy and WTA-WTP ratios

Best estimate selection bias in the value of a statistical life

Cost-benefit analysis of vaccination: a comparative analysis of eight approaches for valuing changes to mortality and morbidity risks

International survey on willingness-to-pay (WTP) for one additional QALY gained: what is the threshold of cost effectiveness?

Conjoint analysis of French and German parents' willingness to pay for meningococcal vaccine

Adolescent values for immunisation programs in Australia: a discrete choice experiment

Cost-effectiveness comparison of monovalent C versus quadrivalent ACWY meningococcal conjugate vaccination in Canada

Meningitis B vaccine to be introduced in UK after U turn on its costeffectiveness

Administration of epinephrine for life-threatening allergic reactions in school settings

Cardiac disease and risk of sudden death in the young: the burden of the phenomenon

Trends in sudden cardiovascular death in young competitive athletes after implementation of a preparticipation screening program

The Joint Committee on Vaccination and Immunisation's advice on extending human papillomavirus vaccination to boys: were cost-effectiveness analysis guidelines bent to achieve a politically acceptable decision? Value Health

Acknowledgments: The authors thank John Forbes for editing assistance.