key: cord-013275-n7sf5ude authors: Drake, Tom; Chalabi, Zaid; Coker, Richard title: Buy now, saved later? The critical impact of time-to-pandemic uncertainty on pandemic cost-effectiveness analyses date: 2013-12-24 journal: Health Policy Plan DOI: 10.1093/heapol/czt101 sha: doc_id: 13275 cord_uid: n7sf5ude Background Investment in pandemic preparedness is a long-term gamble, with the return on investment coming at an unknown point in the future. Many countries have chosen to stockpile key resources, and the number of pandemic economic evaluations has risen sharply since 2009. We assess the importance of uncertainty in time-to-pandemic (and associated discounting) in pandemic economic evaluation, a factor frequently neglected in the literature to-date. Methods We use a probability tree model and Monte Carlo parameter sampling to consider the cost effectiveness of antiviral stockpiling in Cambodia under parameter uncertainty. Mean elasticity and mutual information (MI) are used to assess the importance of time-to-pandemic compared with other parameters. We also consider the sensitivity to choice of sampling distribution used to model time-to-pandemic uncertainty. Results Time-to-pandemic and discount rate are the primary drivers of sensitivity and uncertainty in pandemic cost effectiveness models. Base case cost effectiveness of antiviral stockpiling ranged between is US$112 and US$3599 per DALY averted using historical pandemic intervals for time-to-pandemic. The mean elasticities for time-to-pandemic and discount rate were greater than all other parameters. Similarly, the MI scores for time to pandemic and discount rate were greater than other parameters. Time-to-pandemic and discount rate were key drivers of uncertainty in cost-effectiveness results regardless of time-to-pandemic sampling distribution choice. Conclusions Time-to-pandemic assumptions can “substantially” affect cost-effectiveness results and, in our model, is a greater contributor to uncertainty in cost-effectiveness results than any other parameter. We strongly recommend that cost-effectiveness models include probabilistic analysis of time-to-pandemic uncertainty. Pandemic influenza events have occurred repeatedly throughout history (Potter 2001) . The recent re-emergence of highly pathogenic avian influenza H5N1 in 2003 and the less fatal but more transmissible H1N1 swine influenza strain of 2009 have pushed pandemic influenza up the public health policy agenda. As a consequence funds earmarked for pandemic influenza preparedness have risen markedly over the past decade (United Nations System Influenza Coordinator 2010) . Correspondingly, the number of economic evaluation or costeffectiveness studies of pandemic influenza investment options has also increased. A recent systematic review by Pérez Velasco et al. (2012) indicates that there have been 44 such studies to date. Methodology for the economic evaluation of pandemic preparedness investments, and public health interventions generally, draws on methods developed for health technology assessment. Methodological challenges raised by the more complex decision context of public health investment including pandemic influenza preparedness are still being identified and addressed (Weatherly et al. 2009; Drake et al. 2012) . Uncertainty is a characteristic feature of a pandemic. There is a risk that an as yet unknown highly pathogenic and highly transmissible pathogen will emerge and cause substantial mortality and morbidity worldwide. We do not, and perhaps cannot, know when this will happen, how virulent or how infectious a novel pathogen will be. In general, healthcare economic evaluations make an implicit assumption of consistency in health burden. An analysis based on a clinical trial, for example, is only relevant to policy making if the evaluation context remains reasonably consistent over time. In contrast to this, a pandemic is a one-off event and although there may be similarities to previous pandemics it will nevertheless be fundamentally and unpredictably different. A key challenge for pandemic influenza health economists is to model prospectively; considering how future pandemic events could present differently from past events. Sensitivity and uncertainty analysis are separate but related approaches for assessing the influence of input model parameters on the outcome result. These terms are sometimes used interchangeably but we use the concept of 'sensitivity' to refer to the degree of change in model output for a given change in a model input parameter. In contrast, 'uncertainty' has Bayesian connotations and has been defined as 'any deviation from the unachievable ideal of completely deterministic knowledge of the relevant system' (Walker et al. 2003) . We consider attributable model uncertainty in terms of the total imprecision in the model output due to imprecision in one or more parameter inputs (O'Hagan and Luce 2003; Cooper et al. 2004; Duintjer Tebbens et al. 2008) . Pandemic economic evaluations often consider a single base case pandemic scenario, then explore a small number of variations from this base case using univariate sensitivity analysis (e.g. Eynard et al. 2004; Halder et al. 2011) . Pandemic 'uncertainty' is usually not given special attention and the parameters included in a sensitivity analysis vary between studies with pandemic attack rate, mortality rate and particularly time-to-pandemic, all omitted in different studies to-date. A more robust approach than a simple deterministic sensitivity analysis is to simulate a large number of scenarios, drawing new parameter results for each scenario according to predefined probability distributions. This method is known as 'Monte Carlo' sampling, and in health economics is increasingly used in probabilistic sensitivity analysis. However, Monte Carlo sampling (or its stratified version, Latin Hypercube Sampling) can also be used for uncertainty analysis or even replace the deterministic base case scenario. Many pandemic economic evaluations do take a probabilistic approach, including the first published study of this kind (Meltzer et al. 1999 ). However, many subsequent studies present a single base-case scenario despite the choice of pandemic characteristics being somewhat arbitrary. Time-to-pandemic-does it affect decision making? To a policy maker or public health planner, whether a pandemic occurs in 5 or 50 years makes a big difference in terms of investment choices and resource allocation. In all sectors, including health, the length of time before gains or losses are likely to occur is weighed in the decision making process. In economics, discounting is the practice of adjusting the value of future gains or losses to reflect their present day equivalent. This is applied to costs and, in many cases, to health impact and in theory reflects the time preferences of the population. When undertaking cost-effectiveness analysis of pandemic influenza investment options, there are four general ways to handle pandemic timing: (1) Retrospective-fixed time analysis. (2) Prospective-fixed time analysis. (3) Prospective-deterministic sensitivity analysis. (4) Prospective-probabilistic sensitivity or uncertainty analysis. TIME-TO-PANDEMIC UNCERTAINTY, DOES IT MATTER? Interestingly only a minority of pandemic economic evaluations to date include a sensitivity analysis of time-to-pandemic. Within the pandemic economic evaluations included in the review by Pérez Velasco et al. (2012) , we identify four studies that include pandemic timing in a univariate sensitivity analysis (Meltzer et al. 1999; Siddiqui et al. 2008; Lee et al. 2009; Newall et al. 2010) , and a further three that model uncertainty in pandemic timing probabilistically (Balicer et al. 2005; Lugnér and Postma 2009a,b; Carrasco et al. 2011) . Siddiqui and Edmunds (2008) found that pandemic timing is the second largest source of parameter uncertainty after mortality rate, while Meltzer et al. (1999) find the results and implications for policy 'depend most [our emphasis] on the assumed probability of the pandemic', although they do not include this in the probabilistic analysis. To date, all probabilistic models assume a Poisson stochastic process (a sequence of discrete events where the times between successive events are independent and exponentially distributed), expressed in some studies as a constant annual probability of a pandemic. The implicit assumption is that emergence of a pandemic event has a constant year on year probability driven by random mutation or re-assortment, which eventually produces an influenza strain that results in effective humanto-human transmission and thus has pandemic potential (Webster 1997) . While viral mutation will be important, there are a range of socio-economic, environmental and ecological factors which may influence the emergence of a future pandemic (Jones et al. 2008) . The probability of a pandemic event that arises from these complex factors may not be well characterized by Poisson stochasticity. Pandemic preparedness measures such as stockpiling of antiviral therapies or personal protective equipment require an upfront investment for an unknown benefit at an unknown point in the future. The aim of this study is to establish whether uncertainty in when a pandemic will next occur (henceforth 'time-to-pandemic') is important when evaluating whether pandemic preparedness activities are an efficient use of scarce resources. Is calculation of a cost-effectiveness ratio sensitive to changes in assumptions of time-to-pandemic? Is time-to-pandemic an important driver of total model uncertainty? We explore the importance of this uncertainty when using standard cost-effectiveness methods to evaluate investment in health system pandemic preparedness. We consider a case study of antiviral stockpiling in Cambodia, a low-income country with many pressing demands on healthcare resources considered a hotspot for emerging infectious diseases (Coker et al. 2011) . The comparison is 'no stockpile', and the evaluation perspective is from the medical provider. Health impact is measured in disability adjusted life years (DALYs), and costs are the cost of the antiviral stockpile. Health gains resulting from the stockpile occur at a certain point in the future in the event of a pandemic and are therefore discounted, whereas the investment cost occurs in the present and so is not. This analysis is intended as a stylized decision scenario and is not intended as a cost-effectiveness analysis of antiviral stockpiling in Cambodia. To analyse the impact of time-to-pandemic on model output, we use a simple pandemic event model to generate an incremental cost-effectiveness ratio (ICER) for an antiviral stockpile. For a population ðmÞ a proportion ðhÞ is hospitalized due to the pandemic virus during the course of the pandemic. Of these a further proportion f ð Þ dies from the pandemic disease. A quantity ðqÞ of antivirals are stockpiled at cost ðcÞ; for simplicity the cost of stockpile replenishment is not included. Receiving an antiviral when hospitalized due to pandemic influenza reduces the risk of fatality by a proportion ðrÞ. We define a parameter set which is broadly representative of the Cambodian context. The model structure is outlined as a probability tree in Figure 1 , and all parameters are detailed in Table 1 . The model was coded in the statistical software package 'R', version 2.15.2. The net health impact ðÁÞ in terms of DALYs is calculated by subtracting total deaths in a baseline scenario from total deaths in an intervention scenario and converting to DALYs using a conservative assumption of an average 30 DALYs lost per death. Pandemic costs and consequences are adjusted to present day value using the discount factor (1 þ discount rate) ðÞ and timeto-pandemic ðtÞ. The discounted health impact ðÀÞ is given by: The ICER ð Þ is then given by: This highlights that the ICER is proportional to the discount factor to the power of 'time-to-pandemic', clarifying the interaction between discount rate and pandemic timing. The presence of a power law helps to explain the extent to which uncertainty in discount rate and time-to-pandemic affects the ICER. There is some debate as to whether health outcomes should be discounted (Severens and Milne 2004; Bos et al. 2005) . This debate centres on whether the measure of health utility already incorporates a valuation of time. It seems reasonable that long duration illnesses such as HIV/AIDS do contain consideration of the value of future years. For pandemic or epidemic events where the disease duration is short but when the case occurs is uncertain, it seems likely that some form of discounting is needed to reflect time preferences. That is, given the choice, an individual would prefer to contract a highly pathogenic influenza in 10 years rather than next year. The cost-effectiveness analysis considers the cost per DALY averted for scaling up the antiviral stockpile by 1 million courses. Potentially serving 7% of the 14.3 million total population a stockpile of 1 million courses is a substantial increase on the 75 000 courses currently retained (data supplied by Ministry of Health, Cambodia). To gather data on time-to-pandemic, we considered major pandemics in the last 300 years and calculated the interpandemic period. Information on possible earlier pandemic event years was not included as the veracity of some pandemic Assumed between 0.1% and 5% of the population would be hospitalized at some point during the course of a pandemic. Case fatality proportion f Uniform (0.10-0.70) Lower: WHO severe pandemic threshold is 2% case fatality; however, this refers to the proportion of all cases which die. This is the case fatality proportion for hospitalized patients which we would expect to be higher. We assume a lower bound of 10% and upper bound of 70% case fatality. Time-to-pandemic (years) t Uniform All fitted to data in Table 2 dates is disputed (Potter 2001 ) (pandemic dates and intervals are presented in Table 2 ). Using the historical pandemic interval data we carry out a univariate sensitivity analysis to assess whether observed differences in the number of years between pandemics can substantially affect cost-effectiveness results. However, univariate analysis is limited because it does not account for interaction between the values of other model parameters. Sensitivity to time-to-pandemic may be different given a different set of other input parameter values. We therefore use multivariate Monte Carlo simulation to sample parameter values from uniform distributions of six model parameters (time-to-pandemic, case fatality proportion, hospitalization proportion, discount rate, intervention cost and intervention effectiveness), repeating 20 000 times to produce results from a wide range of parameter combinations. Parameter 'importance' is assessed using elasticity and mutual information (MI) measures (described below). The purpose of this study is to determine whether uncertainty in the number of years before a pandemic occurs (time-topandemic) is important in cost-effectiveness analysis. For this reason, the analysis is designed as a 'challenge' model using large estimates of uncertainty in other model parameters to present the greatest challenge to the importance of time-topandemic. In most cases, little data exists to fit sampling distributions of the model parameters so in general uniform distributions have been used with ranges at logical or plausible limits. If pandemic timing uncertainty is shown to be important compared with large estimates of uncertainty in other pandemic characteristics then we can conclude that pandemic timing uncertainty is important to cost-effectiveness analysis of pandemic influenza preparedness. We use two measures to determine the importance of uncertainty in the time to the pandemic to the overall uncertainty in ICER: mean elasticity and MI. Although the potential value of the MI measure in health economics has been reported (Coyle et al. 2003) , it is not commonly seen in the health economics literature. Similarly, while elasticity is a concept commonly used in economics it is rarely found in healthcare economic evaluation. We describe both measures fully here for completeness. The elasticity of a parameter describes the extent to which an output changes given an incremental change in an input parameter. This is frequently used in economics to describe effect of demand given a change in price. Using elasticity to measure parameter importance is a logical step from routine sensitivity analysis of economic evaluation, which commonly presents a change in ICER given fixed univariate changes in an input parameter. Standard univariate sensitivity analysis is useful in quantifying the impact of possible parameter changes on cost effectiveness. To compare the sensitivity of the ICER to each parameter on equal basis, we normalize the univariate sensitivity analysis, producing a point elasticity for each Monte Carlo simulation. We take the mean of the point elasticities and report the mean parameter elasticity as a unit-less measure of parameter importance. The ICER is a function of the parameters listed in Table 1 . In a general mathematical formulation if we denote the ICER by Y and the model parameters by ðX j , j ¼ 1 . . . nÞ where n is the total number of parameters, then the ICER can be represented as a non-linear function g of the n parameters: The normalized local elasticity measure is defined as: where the symbol @ denotes partial differentiation, and the elasticity is evaluated at X j ¼ x. È Y, X j À Á is the proportional change in Y due to a small proportional change in X j . The mean elasticity measure " È Y, X j À Á is more appropriate to assess parameter sensitivity under uncertainty (where there is not a high degree of confidence in the expected value). This is given by Coyle et al. (2003) : where n is the number of point-wise evaluations carried out within the range X j, min , X j, max  à , and the symbol means 'element of'. MI analysis is based on information and entropy theory (Fraser and Swinney 1986; Veyrat-Charvillon and Standaert 2009; Batina et al. 2011) , the origins of which lie in estimating the entropy of thermodynamic systems. Its use has been advocated in the health economics literature (Coyle et al. 2003) . The entropy HðYÞ of an uncertain variable Y (such as the ICER) is defined in continuous space by the following integral equation: where ðYÞ is the probability density function of Y and ½Y min , Y max is the range of Y (i.e. Y min and Y max are the lower and upper bounds of Y, respectively). The MI between the ICER ðYÞ and a variable on which it depends X j (e.g. case fatality proportion or time-to-pandemic) is defined by the following double integral equation: where the double integration is carried over the range of ICER and the range of parameter X j , X j, min , X j, max  à and Y, X j À Á is the joint probability density function of Y and X j . ÉðY, X i ) is the MI and can be interpreted as the amount of uncertainty that is reduced in Y if the uncertainty in X i is removed (e.g. because of new information acquired on X i ) or alternatively the amount of information that Y and X j share. In practice, discrete rather than continuous analysis is carried out. The counterpart equations to (6) and (7) in discrete space are: and where pðY ¼ yÞ denotes the probability that Y ¼ y, and p Y ¼ y, X j ¼ x À Á denotes joint probability that Y ¼ y and X j ¼ x. From Equation (9), it can be shown mathematically (Dragomir and Goh 1996) that ðY, X j Þ ! 0, Y, X j À Á ¼ 0 if Y is independent of X j and that ðY, X j Þ attains its maximum when Y ¼ X i (i.e. Y and X j are the same) or when Y is a deterministic function of X j . This fits exactly with the interpretation given above of MI. In one extreme, if the ICER is independent of a parameter then reducing the uncertainty of that parameter would not have any effect on the uncertainty of the ICER. At the other extreme, if the ICER depends on one parameter only, then reducing the uncertainty in the ICER is maximized by reducing the uncertainty of the parameter to its minimum. We used the R package 'entropy' for MI calculations (Hausser and Strimmer 2009 ). Variation in ICER and parameter elasticity by parameter value. In the parameter uncertainty analysis outlined above a uniform distribution is used to model time-to-pandemic uncertainty (mean 40 years, range 14-66 years). We relax this assumption in a secondary analysis of uncertainty by also running analyses with normal, Poisson and gamma distributions. All distributions are fitted to the data on pandemic interval in Table 2 , using the fitdistrplus package within R (Delignette-Muller et al. 2012). The normal distribution is truncated at 0 to prevent sampling of negative values. We use the MI to assess whether time-to-pandemic uncertainty remains important irrespective of sampling distribution choice. The World Health Organization recommends using a costeffectiveness threshold of 1-3 times the national Gross Domestic Product (GDP) per capita (World Health Organization 2012); however, there are many limitations to using this rule including situations when funds originate from international organizations (Drake 2013 ). Nevertheless, we include an illustrative calculation of the proportion of simulations found to be 'cost effective' at a threshold of US$900 per DALY averted, the GDP per capita in Cambodia (World Bank 2012). The mean cost of the 1 million course stockpile is US$9 million (median also US$9m) with mean 831 000 DALYs averted (median: 235 000 DALYs averted), and a mean ICER of US$28 375 per DALY averted (median: US$911 per DALY averted). Note that the mean and median ICER is calculated per Monte Carlo simulation not from the mean and median costs and DALYs averted. Figure 2 shows the spread of simulation results on the cost-effectiveness plane. Costs range uniformly between US$3 and 15 million while deaths averted are highly skewed with a small number of pandemic scenarios producing a high death toll. Univariate analysis of change in ICER adjusting for timing according to pandemic intervals from historical data is presented in Table 2 ; the range is US$112 to US$3599 per DALY averted. In the probabilistic analysis, parameter sensitivity and uncertainty are assessed by calculation of elasticity and MI, respectively (Table 3) . Using both measures, the most 'important' parameters identified are discount rate and time-topandemic. These have an elasticity slightly greater than two, indicating that, on average, an increase in discount rate or time-to-pandemic causes a greater than proportional increase in the ICER. All other parameters have a mean elasticity magnitude of 1 indicating that a change in the parameter causes a roughly proportional increase or decrease (for negative elasticities) in the ICER; that is an doubling of the parameter value causes the ICER to double or halve, respectively. Similarly, MI scores for discount rate and time-to-pandemic are two to three times larger than the other parameters, indicating a greater contribution to total uncertainty in the ICER result. Figure 3 presents how ICER and elasticity vary with each parameter. The first row of Figure 3 shows how change in input parameter (x-axis) affects the ICER result (y-axis). Each point represents a model simulation and skewness in the distribution of simulations reflects the relationship between the parameter and the ICER. A loess mean regression line with 95% confidence interval is overlaid using the R package 'ggplot' (Wickham 2009 ). The second row of Figure 3 also plots change in input parameter (x-axis) but this time against elasticity, outlining whether parameter point elasticity is dependent on the value of the parameter being measured or on other parameters. Elasticity is measured for each Monte Carlo simulation, and each simulation probabilistically samples from six model parameters, so each elasticity measurement corresponds to a new model parameter set describing a new pandemic scenario. For the hospitalization proportion and the case fatality proportion elasticity is constant at À1 irrespective of their value at measurement or the value of other parameters. Similarly, intervention cost has a constant elasticity of 1. Elasticity for intervention effectiveness increases at greater parameter values but is independent of other model parameter values. The elasticity for discount rate and time-topandemic is dependent not only on their own parameter value but also on the value of the other parameter. That is, there is an interaction between discount rate and time-to-pandemic when assessing elasticity. The uniform sampling distribution for time-to-pandemic is replaced with gamma, normal and Poisson distributions, repeating 20 000 Monte Carlo simulations for each distribution type. Mean ICERs for Uniform, Gamma, Normal and Poisson distributions are US$28 375, US$78 809, US$33 006, US$10 199 per DALY averted, respectively (Table 4 ). The proportion of Prior for uniform, gamma, normal and Poisson time-to-pandemic sampling distributions and corresponding ICER output distribution. simulations cost effective at a willingness-to-pay of US$900 per DALY averted does not differ substantially with different timeto-pandemic sampling distributions and the contribution of time-to-pandemic to uncertainty in the ICER remains high compared with other model parameters as measured by the MI. This can be seen in Figure 4 which presents the prior (time-topandemic) and posterior (ICER) probability density curves. The shape of the posterior or ICER probability density curve is consistent irrespective of the prior or time-to-pandemic distribution choice. The results of this analysis demonstrate that uncertainty in time-to-pandemic is important in pandemic cost-effectiveness models. Both univariate and multivariate measures of elasticity found that in our model the ICER was most sensitive to changes in time-to-pandemic and discount rate. Similarly, MI analysis found that time-to-pandemic and discount rate were the principle drivers of uncertainty in the model results. Uncertainty in pandemic hospitalization rate was also important, as would be expected. While time-to-pandemic is important the choice of sampling distribution was less so, with similar contributions to uncertainty resulting from uniform, gamma, normal and Poisson distributions. Pandemic cost-effectiveness models should either take a probabilistic (Bayesian) modelling approach to incorporate uncertainty in time-to-pandemic and other pandemic parameters into the base case analysis or conduct a full probabilistic sensitivity analysis around a fixed base case scenario. The large amount of pandemic uncertainty in the model, which cannot easily be reduced, may mean that reducing uncertainty in other parameters, such as treatment effectiveness, offers minimal improvement to the usefulness of pandemic cost-effectiveness models. One additional aspect of the relationship between time-topandemic and cost effectiveness is that if year-on-year no pandemic occurs then the expected time to pandemic decreases and the expected cost effectiveness of investment rises. Ironically there may be an inverse relationship between cost effectiveness and likelihood of investment in pandemic preparedness measures, i.e. decision makers are more likely to act on pandemic preparedness immediately following a pandemic and interest wanes with time. The consequence of discounting future health losses leads to a greater emphasis on the short term. Should standard rate of time preference be applied to events with potentially catastrophic impact? Are population time preferences in line with individual preferences? Individual preferences can betray the public good as seen in individual vs social demand curves for vaccine uptake (Fine and Clarkson 1986) . If we take a long run perspective, individuals come and go, and so they have a clear timepreference, but the population remains. In effect the problem becomes philosophical; relating to the current generation's responsibility to future generations. This problem is mirrored in other population level challenges such as climate change, national debt accumulation or risk of redundant antibiotics resulting from widespread antibiotic usage. This analysis points towards the heavy impact that time-preferences of individuals has on these decision contexts. Further work is needed to (re)establish norms for discounting in these contexts. Central to this will be the question of whether time preferences are different when individuals make decisions as a community, whether time preferences differ with very high impact events and whether discount rates are indeed constant for long time-spans. This study has several limitations. The model is a simple decision tree and does not include more sophisticated model structures recommended in pandemic preparedness cost-effectiveness analysis (Lugnér and Postma 2009a,b; Drake et al. 2012) . However, this is not likely to be a significant problem as these approaches aim to more accurately assess the effectiveness of pandemic preparedness of interventions and therefore would be unlikely to lead to a lower estimate of the importance of time-topandemic. The model also only considers antiviral stockpiling as a pandemic preparedness measure. Other pandemic preparedness options may not require upfront investment, such as social distancing measures in response to a pandemic and therefore time-to-pandemic would not be an issue. Pandemic preparedness requires upfront investment and the benefits of this investment are only realized during the course of the next pandemic, an unknown number of years after the investment. We find that in the economic evaluation of pandemic preparedness investments the uncertainty in timeto-pandemic is a dominant factor in model uncertainty. Mathematical models to assess the cost effectiveness of pandemic preparedness options should include probabilistic sensitivity or uncertainty analysis of time-to-pandemic. At the same time, steps should be taken to carefully consider the most appropriate approach to discounting the impact of future catastrophic events such as pandemics. Cost-benefit of stockpiling drugs for influenza pandemic Mutual information analysis: a comprehensive study Discounting health effects in pharmacoeconomic evaluations: current controversies Strategies for antiviral stockpiling for future influenza pandemics: a global epidemiceconomic perspective Emerging infectious diseases in southeast Asia: regional challenges to control Comprehensive decision analytical modelling in economic evaluation: a Bayesian approach Measures of importance for economic analysis based on decision modeling fitdistrplus: to fit a parametric distribution to non-censored or censored data A counterpart of Jensen's discrete inequality for differentiable convex mappings and applications in information theory Priority setting in global health: towards a minimum DALY value. Health Economics (early view online Cost-effectiveness analysis of pandemic influenza preparedness: what's missing? Uncertainty and sensitivity analyses of a dynamic economic evaluation model for vaccination programs Influenza pandemic planning in Switzerland-an economic perspective Individual versus public priorities in the determination of optimal vaccination policies Independent coordinates for strange attractors from mutual information Cost-effective strategies for mitigating a future influenza pandemic with H1N1 2009 characteristics Entropy inference and the James-Stein estimator, with application to nonlinear gene association networks Global trends in emerging infectious diseases Economic analysis of pandemic influenza vaccination strategies in Singapore Investment decisions in influenza pandemic contingency planning: cost-effectiveness of stockpiling antiviral drugs Mitigation of pandemic influenza: review of cost-effectiveness studies The economic impact of pandemic influenza in the United States: priorities for intervention Cost-effectiveness of pharmaceutical-based pandemic influenza mitigation strategies A Primer on Bayesian Statistics in Health Economics and Outcomes Research Systematic review of economic evaluations of preparedness strategies and interventions against influenza pandemics A history of influenza Discounting health outcomes in economic evaluation: the ongoing debate Cost-effectiveness of antiviral stockpiling and near-patient testing for potential influenza pandemic Animal and Pandemic Influenza: A Framework for Sustaining Momentum Mutual information analysis: how, when and why Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support Methods for assessing the cost-effectiveness of public health interventions: key challenges and recommendations Predictions for future human influenza pandemics ggplot2: Elegant Graphics for Data Analysis World Bank World Health Organization The authors would like to thank Korbinian Strimmer for advice on calculating MI using the R 'entropy' package and Yoel Lubell for comments on the study aims and methods. The work was supported by the German Federal Ministry for Economic Cooperation and Development (BMZ) as part of Deutsche Gesellschaft für Internationale Zusammenarbeit's (GIZ) German Pandemic Preparedness Initiative. Ethical approval for the wider project has been granted by the Ministry of Health in Cambodia and the London School of Hygiene and Tropical Medicine. No empirical data was collected for this study. This manuscript has not been published in whole or in part elsewhere. TIME-TO-PANDEMIC UNCERTAINTY, DOES IT MATTER? None declared.