key: cord-0852285-c4gzw1f9
authors: Herida, Magid; Dervaux, Benoit; Desenclos, Jean-Claude
title: Economic Evaluations of Public Health Surveillance Systems: a Systematic Review
date: 2016-02-05
journal: Eur J Public Health
DOI: 10.1093/eurpub/ckv250
sha: 158c03781929cabe37835cb32ec1156cbc32b3f5
doc_id: 852285
cord_uid: c4gzw1f9

nan

' P ublic Health surveillance is the ongoing systematic collection, analysis and interpretation of health data, closely integrated with the timely dissemination of these data both to those providing the data and to those who can apply the data to control and prevention programs'. 1 Public health surveillance is therefore, closely linked to action as it provides accurate and validated information to local and national health authorities in order that these same authorities can implement appropriate prevention and control measures and health promotion strategies. 2 This definition was set up several decades ago. Since then, the adoption of the revised International Health Regulations in 2005 has implemented new regulations such as national obligations to guarantee a set of core surveillance and response capacities to prevent international spread of disease irrespective of its origin (biological but also chemical or radio-nuclear sources). 3 Furthermore, new diseases have emerged (H1N1pdm09, Middle East Coronavirus), some others have spread dramatically such Ebola virus disease in West Africa. 4 In the context of globalisation, the concept of public health surveillance has evolved to include public health security. 5, 6 Politicians and stakeholders rely upon their public health agencies to be informed in a timely fashion and to be able to respond promptly to all potential health treats that may arise as populations need protection whatever the risk. For public health surveillance, this implies the need to monitor a very large spectrum of all hazards health threats. At the same time, as many public agencies are currently facing budgetary constraints, they must allocate theirs resources as efficiently as possible, as surveillance systems cannot be implemented for all known health threats. The priorities for diseases surveillance need to be reviewed regularly in order to guarantee that the most topical public health issues are consistently tackled and to ensure quality and effectiveness of surveillance. Furthermore, best practice recommends periodic evaluation of public health surveillance systems (PHSS). 7 Like other national public health agencies, 8 the French National Institute of Public Health (InVS) has implemented several prioritization exercises to guide its own surveillance strategy. [9] [10] [11] It has also implemented an external and formal evaluation process of its surveillance systems, based on a generic evaluation protocol drawn up in accordance with the United States Centers for Disease Control and Prevention (CDC) guidelines, 12 and using a design developed by the European Center for Diseases Prevention and Control (protocol provided by Dr A. Ammon, personal communication). Four surveillance systems for infectious diseases were assessed between 2009 and 2013. 13 Given the current economic context, this assessment process was less than comprehensive as it did not consider the increasingly important question of the costs and the benefits of surveillance systems. The CDC guidelines 12 indicate that costs should not be estimated alone but be judged relative to benefits or at least relative to the effects of the surveillance system on prevention, care, research or decision-making. The guidelines state also that 'a more realistic approach would be to judge costs on the objectives and usefulness of the system'. However, the CDC guidelines do not clearly define methods to do so. A WHO report issued in 2005 underlined that the evaluation of the costs and the benefits of national surveillance and response systems was an area of great importance. In the report, the authors indicated that little related work has been carried out to that point and highlighted a number of methodological difficulties in evaluating benefits of a surveillance and response system such as implementing the appropriate study design, defining the scope of the surveillance or assigning a monetary value to health outcomes averted. 14 To assess the current development of economic evaluations in the field of public health surveillance, we performed a systematic review of the literature. In this review, we also assessed the methods used in the selected studies to evaluate the benefits of public health surveillance.

For this systematic review, we followed PRISMA methodology PRISMA checklist for this review is available as supplementary material on line. 15 The search was performed in two major databases for health interventions Medline and Scopus. In each database, two search strategies were performed. The first search strategy used in each of the database was to employ a structured thesaurus in order to focus the search. The second search strategy in each database use text words (in the title and in the abstract for Medline and in the title for Scopus) in order to capture articles not yet or incorrectly indexed. We focused on two major topics: public health surveillance and full economic evaluations. Public Health surveillance is included in the MESH term 'population surveillance' and full economic evaluations [cost-benefit analysis (CB), cost-effectiveness analysis (CE) and cost-utility analysis (CU)] were included in the MESH term 'cost-benefit analysis'. The four search strategies are displayed in Appendix 1.

Additionally, a search using terms 'Surveillance' AND ('CB analysis' or 'CE analysis' or 'CU analysis') in any field was conducted in the CRD's NHS Economic Evaluation Database (NHS EED).

Only studies published in peer-reviewed literature which performed a full economic evaluation of the PHSS in question were included in the present review. A full economic evaluation is defined as 'the comparative analysis of alternative course of actions in terms of both their costs and consequences'. 16 Three main types of full economic evaluation were considered eligible: cost-effectiveness analyses, cost-utility analyses and cost-benefit analyses. The research was conducted on relevant articles published in English or French written articles until December 2013, and no time period was specified.

Cost-minimization analyses were excluded as the consequences or the benefits of compared surveillance systems were assumed to be equivalent and only relative costs were compared.

PHSS evaluations on technical attributes other than costs and benefits (sensitivity, timeliness, data quality. . .), cost evaluations of a PHSS, comparison of costs between two types of PHSS, comparison of performance indicators between two types of PHSS were excluded of the analysis as they do not provide any evaluation of the benefits of the surveillance. Besides, in order to get insights in a reproducible methodology for public health surveillance, evaluations of surveillance systems in the field of veterinary surveillance were excluded of this review. Finally, articles written in a language other than French or English were excluded.

A first review on the title and the abstract was performed to identify and exclude articles written in a language other than English or in French, and those related to veterinary surveillance. The remaining retrieved papers were eligible for a full text review. The first author selected the studies which met the inclusion criteria. When there was any doubt about whether to include or not the article, the latter was reviewed by the second author and discussed with all the authors to reach a decision.

The two search strategies on Medline provided 407 articles of which 5 were duplicates leading finally to 402 papers selected in this first database. The research on the second database (Scopus database) identified 556 articles. Overall, the searches in the two databases produced 958 articles, of which 343 were duplicates and therefore excluded. Finally, the four search strategies in the two databases led to 615 original articles (figure 1).

After a first review on the title and abstract of the 615 articles, 21 papers written in a language other than English or French and 27 other articles related to veterinary surveillance were excluded. Finally, 567 papers were eligible for a full text review.

Among the 567 papers selected, 461 were not full economic evaluations and were therefore excluded while 106 included full economic evaluations. The majority of the latter assessed medical screening treatments (including vaccination strategies) (n = 80). Among the 26 remaining articles, 17 addressed public health interventions and finally only 9 full economic evaluations of a surveillance system were retrieved and therefore selected according our review's eligibility criteria. [17] [18] [19] [20] [21] [22] [23] [24] [25] Among these nine selected studies, eight were identified in the two databases and one 17 was only identified in the Scopus database. The references of the selected studies were also checked and no additional study was found. Two of the nine studies 21,25 referenced one of the other seven selected studies. 19, 22 The query performed in the NHS EED identified three articles already identified from the two above-mentioned databases. [22] [23] [24] Summary of the selected studies Most of the items listed in the CHEERS checklist were used to assess the method and the results of each of these studies looking at target population, setting and location, study perspective, time horizon, discount rate, choice of model, choice of health outcomes, estimation of costs and resources, currency and conversion and incremental costs and outcomes (tables 1 and 2). 26 The nine selected studies accounted for only 1.5% of the 567 references initially identified. They were published between 1986 and 2010. Five of the nine topics studied infectious diseases surveillance (HIV infection, salmonella outbreak, meningitis, hepatitis A and Escherichia coli (EC O157:H7) infection). 18, 19, 21, 22, 25 Three addressed environmental exposure (heat waves, professional asthma and anthrax exposure) 17,23 and one described the benefits of the implementation of an electronic surveillance system. 20 These evaluations were conducted in the USA (n = 5), in UK (n = 3) and in the Burkina Faso (n = 1). There were three cost-effectiveness, 21, 24, 25 one cost-utility 23 Medical costs (local and regional level)

The C/E ratio was estimated at $23/per case averted and at $98/meningitis-related death averted Both C/E ratios were cost-saving Target populations varied among the studies from a whole country or a federal state 22,25 to a single county or city. 17, 18 Two studies were performed on theoretical exposed populations from 100 to 100 000 persons exposures. 23, 24 Time period was mentioned for four studies and costs were adjusted to the study period in all four. [22] [23] [24] [25] Three of the studies were based on retrospective events lasting from 22 weeks to 3 years. [17] [18] [19] The two remaining studies were based on one year surveillance periods. 20, 21 With respect to the four cost-effectiveness studies, two costeffectiveness and the one cost-utility analysis, compared two surveillance strategies (active case finding versus passive surveillance or surveillance and response versus no surveillance). 23-25 A Markov model was used for two of these studies. 23, 24 An experimental design was set up for the third one where the authors used the surveillance data from the previous 7 years' data for 5 years after the implementation of the surveillance and response program in order to assess the net costs for each health outcome prevented. 25 Health outcomes were expressed in number of cases or death prevented for two studies and in number of Qalys saved for one. These three studies have estimated an incremental cost-effectiveness (or cost-utility) ratio for the more efficient strategy. The fourth cost-effectiveness study focused on HIV surveillance. The authors of that study were not able to estimate the number of cases prevented by the HIV surveillance activities alone (which constituted only a part of the entire HIV prevention program). To overcome this methodological issue, the authors defined a level of cost-effectiveness based on a median value of ratios of 116 cost-effectiveness published studies. Based on this ratio, they estimated the number of HIV infections the entire HIV prevention program would need to prevent in order for it to be considered cost-effective. 21 With respect to the five cost-benefit studies, two were based on a comparison of two surveillance systems (active versus passive and electronic reporting versus manual reporting). Benefits were estimated as the costs saved for the cases prevented for one study and by the time gained in employing electronically captured surveillance data for the other. 18,20 A third study described the surveillance and response activities implemented to control an outbreak of salmonella. Benefits were estimated by the costs avoided of resulting potential cases and associated deaths if the outbreak had not been controlled by surveillance and response. 19 In the fourth study, the authors estimated the threshold number of cases of Escherichia Coli O157:H7 (EC O157:H7) that the system would need to prevent in order for this system to be considered cost-beneficial. 22 The fifth and final study modeled, over a 3-year period, the number of deaths prevented among persons aged 65 years and over during the days when a heat wave surveillance system had issued a warning. 17 The type of costs included was heterogeneous between the nine studies and depended on the scope of surveillance and response activities considered and the perspective used. The perspective used for the cost analysis was mentioned in five of them. 19, 21, 22, 23, 25 The four remaining in which the perspective was not stated, have used a government-funded public health perspective as they included only the costs linked to the surveillance system and response. 17, 18, 20, 24 Accordingly, some authors included only the direct costs for operating the surveillance system (capital costs, salaries or training costs, recurrent costs) 20,21 while others also included indirect costs linked to treatment and care or those linked to biological testing. 18, 22, 24, 25 Some authors also included indirect costs related to loss of productivity or pain and suffering as they use a societal perspective. 19, 23 In one of the five cost-benefit studies 22 and in three of the four cost-effectiveness studies, 21,23,25 a discount rate was applied. Among the five modelling studies, four have performed sensitivity analyses. [22] [23] [24] [25] Among the five cost-benefit studies, four concluded that a positive benefit existed for active surveillance versus passive or versus no surveillance in term of money or time spent. [17] [18] [19] [20] In the remaining study, the authors considered that the surveillance system was costbeneficial if five infections were averted per year. This estimate is much lower to than the 90 cases of EC O157 reported to the surveillance system the year of the study. 22 Among the four cost-effectiveness and cost-utility studies, two have considered that the implemented surveillance was cost-saving. 23, 25 The third one found different cost-effectiveness ratios under different scenarios of anthrax exposure (probability and number of persons exposed). In one over the four scenarios, the surveillance and emergency response was cost-effective. 24 For the fourth study focusing on HIV surveillance, the entire HIV prevention program was estimated to be cost-effective if at least 333 new cases of infections were averted each year which represents 13% of the new diagnosis reported the year of the study. 21 

Although the efficiency of public health surveillance has been under the scrutiny by governments for many years, this systematic review clearly indicates that, to date, very few economic evaluations of PHSS have been performed worldwide. This observation was already pointed out in 2005 in the above mentioned WHO report. 14 Another systematic review on surveillance system evaluations published in 2012 found only three articles which aimed at evaluating cost-effectiveness of PHSS. 27 Only of the three was selected in this review. 21 The other two articles compared costs or costs and performance indicators, respectively among two surveillance systems and therefore did not evaluate surveillance benefit. 28, 29 For this reason, these two studies did not meet our eligibility criteria. The current review did not find any published studies on this subject from 2010 onwards.

Three main approaches, -cost-effectiveness, cost-utility and costbenefit analyses-were used for economics evaluations of PHSS in the studies selected in this review. Studies focusing on surveillance system's cost-effectiveness or cost-utility provide valuable information for comparing several surveillance strategies and ranking these same strategies among other medical or public health interventions. 30 However, the benefit measured in these studies was limited to the number of health outcomes prevented by one surveillance strategy compared with another one.

The cost-benefit analysis studies reviewed here, all evaluated the benefits of surveillance by assessing the number of cases or deaths the particular system prevented, assigning a monetary value per case prevented or life saved. Classically, if the expenditures for installing and operating the surveillance system are equal to or less than those for treating patients then the system is considered cost-beneficial. 31 This approach implies not only knowledge of the costs of the disease in both its acute phase and for chronic disease over the long term, but also a consideration of all clinical presentations from the mildest to most severe. Besides, assigning a monetary value to a live saved implies obtaining this information from the literature or implementing an ad hoc study.

Estimating the costs of a health intervention requires a standardized approach and implies the need to follow certain methodological steps such as defining the perspective, including direct and indirect costs, discounting future costs and health outcomes, describing all the resources used and assigning a monetary value to these resources. 32 In this review, discrepancies were found between the different studies examined with respects to the type of costs included and with the definition of scope of the surveillance and/or response activities they considered. This highlights the practical obstacles to making standardized costs estimations of PHSS. The above mentioned WHO report 14 suggests including both the costs of surveillance and response as the ultimate purpose of surveillance information is to inform public health decision-making and to guide response and control activities. A conceptual framework of public health surveillance and action comprising six surveillance core activities such as detection, confirmation or analyses and two public health actions (acute and planned responses) and four support activities such as communication and training has recently been developed. 33 This framework aims at assessing the performances of PHSS and could also be useful for conducting standardized cost analysis for PHSS.

In this review, several of the studies examined included surveillance and response activities where the authors estimated the benefits of surveillance through the number of cases or deaths prevented by response activities. Nevertheless, this approach cannot be applied when the primary objective of the surveillance activities is not linked to immediate response as illustrated in the study of HIV cost-effectiveness. Although information coming from surveillance is crucial to guide prevention activities and to allocate funds, in that study the authors underlined their difficulties in quantifying the surveillance-specific benefits of the surveillance from all the benefits provided by the a wide range of prevention activities implemented in the specific public health HIV program. 21 With respect to the CDC guidelines mentioned above for evaluating PHSS, 12 costs and benefits of a surveillance system should be estimated in the light of many other criteria including the objectives and the utility as well as several technical performance attributes. The methods used in the selected studies did not take into account all these parameters and could therefore have led to an underestimation or an overestimation of the benefits of the surveillance if, for example, the objectives were not been reached or if some technical attributes such as timeliness did not meet the quality standards. Finally, assuming that benefits can be defined solely in term of the costs of the health outcomes prevented may constitute a limited approach as cost-benefit analysis of PHSS includes also an option value and the need to take into account positive nonmonetary effects like the perception of health security where the general public is aware of the fact that health threats are regularly monitored and are controlled in a timely and efficiently manner irrespective of the number of cases avoided. There are also economic benefits in preventing diseases or large outbreaks in that people to continue to work and to be productive.

These methodological drawbacks and difficulties listed above may to some extend explain why so little work has been carried out to date in the field of economic evaluation of surveillance systems. In the light of the limitations of commonly used approaches, other methods of economic approaches may be useful. Among these, conjoint analysis (CA) could overcome to some extent the limitations and methodological issues discussed above. The theory behind this approach is that any product or good or service can be described by a set of characteristics and that the extent to which an individual places value on a product is determined by the level of these characteristics. 34 CA asks individuals to state a preference by presenting competing scenarios with both desirable and undesirable characteristics of a product or a service. CA has been applied to health care and public health interventions such as HIV vaccine and cancer control strategies. 35, 36 More recently, CA was performed to prioritize zoonotic diseases of public health concern. 37 CA has been also used to derive utility weight for QALY or to estimate willingness to pay for CB analysis. 38 To our knowledge, the method has not been used for a PHSS and may be worth exploring. Indeed, this type of economic evaluation could help policy makers and health professionals elicit preferences and help estimate their willingness to pay for a surveillance system according to different system characteristics and performance. This technique may also be useful for estimating the non-use value of a surveillance program. 39 This information could be useful to public health agencies and could help them to prioritize surveillance in a context of limited resources.

This review has several of its own limitations. First, because 'Public Health surveillance' is a specific term in Medline, we had to use the generic MESH term 'Population surveillance'. Consequently, many articles were related to medical interventions rather than to public health surveillance and the selected studies accounted for a very small percentage of the identified articles.

Conversely, had we used a query with the term 'Public health surveillance' we would have identified only seven references, none of which being one of those finally selected. Additionally, focusing on public health surveillance evaluation and displaying the different types of studies (economic versus non-economic) led to a much higher number of articles (over 1200 articles) and did not retrieve the selected studies. However, the fact that this review is extensive and that the majority of the selected studies (8/9) were identified in both databases proves that few economic evaluations of a PHSS have been performed to date and that the risk of having missed an article is most probably very low.

Second, the line between public health surveillance and medical intervention or control intervention is often a very thin one and initially some of the studies finally selected [23] [24] [25] were the subject of discussion between the authors of this present review in order to decide whether or not they did in fact deal with surveillance. Third, this article did not aim to critically review the quality of the selected studies. Certain methodological issues are pointed out in this article to underline how difficult it is sometimes to perform a full economic analysis in the field of public health surveillance. Finally, our primary objective was to look at the methods used in the economic evaluation of a PHSS. We did not focus on the results of these economic analyses despite the fact that in most of these studies, the authors considered the surveillance system in question either cost-effective or cost-beneficial.

In conclusion, this review demonstrates that few studies have been performed to evaluate the benefits, a PHSS can bring. The most frequent approach employed to assess these benefits is to value the health outcomes prevented by the surveillance and response activities or to estimate the costs per number of health outcomes avoided by comparing two surveillance strategies. By using these approaches, other dimensions are overlooked. As an alternative, using other economic methods such as conjoint analysis could show promise for future development.

Supplementary data are available at EURPUB online.

The science of public health surveillance

Surveillance et veille: concepts, défintions et organisation

World Health Organization. International health regulations

Ebola in West Africa: from disease outbreak to humanitarian crisis

Exploring the international arena of global public health surveillance

Comprehensive effective and efficient global public health surveillance

Principles & Practice of Public Health Surveillance

Prioritisation of infectious diseases in public health: feedback on the prioritisation methodology

Orientations pour la révision des modalités de surveillance des maladies transmissibles en France (Orientations for revising infectious diseases surveillance systems in France)

Morbidite´et mortalite´dues aux maladies infectieuses d'origine alimentaire en France (Morbidity and Mortality Related to Infectious Disesases in France)

Impacts sanitaires du changement climatique en France. Quels enjeux pour l'InVS? (Health Impacts of Climate Change in France. What are the Challenges for the InVS?)

Centers for Disease Prevention and Control. Updated guidelines for evaluating public health surveillance systems: recommendations from the guidelines wrorking group

Evaluation de différents systèmes de surveillance à partir d'un protocole générique (Evaluation of various public health infectious diseases surveillance systems based on a generic protocol). Revue d'Epide´miologie et de santeṕ ublique

World Health Organization. Evaluating the Costs and Benefits of National Surveillance and Response Systems: Methodologies and Options. Geneva: World Health Organization

The PRISMA statement for reporting systematic reviwes and meta-analyses of studies that evaluate heatlh care interventions: explanation and elaboration

Me´thodes d'e´valuation e´conomique des programmes de sante´

Heat watch/warning systems save lives: estimated costs and benefits for Philadelphia 1995-98

Benefit-cost analysis of active surveillance of primary care physicians for hepatitis A

Economic impact of a nationwide outbreak of salmonellosis: cost-benefit of early intervention

The introduction of a computerized system for notification and improved analysis of infectious diseases in Sheffield

The costs and effectiveness of surveillance of communicable disease: a case study of HIV and AIDS in England and Wales

Costs and benefits of a subtype-specific surveillance system for identifying Escherichia coli O157:h7 outbreaks

Surveillance for isocyanate asthma: a model based cost effectiveness analysis

The cost-effectiveness of strategies to reduce mortality from an intentional release of aerosolized anthrax spores

Modeling the cost-effectiveness of the integrated disease surveillance and response (IDSR) system: meningitis in Burkina Faso

Consolidated health economic evaluation reporting standards (CHEERS) statement

Evaluation of animal and public health surveillance systems: a systematic review

Evaluation of program performance and expenditures in a report of performance measures (RPM) via a case study of two Florida county tuberculosis programs

Comparative analysis of two alternative models for epidemiological surveillance in the Mexican Malaria Control Program

Cost-effectiveness analysis as a guide to resource allocation in Health: role and limitations

Public health surveillance: a tool for targeting and monitoring interventions

Me´thodes d'e´valuation e´conomique des programmes de sante´

Conceptual framework of public health surviellance and action and its application in health sector reform

Using conjoint analysis to elicit preferences for health care

Stakeholder perspectives on European priorities for comprehensive liver cancer control: a conjoint analysis

HIV vaccine acceptability among communities at risk: the impact of vaccine characteristics

A quantitative and novel approach to the prioritization of zoonotic diseases in North America: a public perspective

Conducting discrete choice experiments to inform healthcare decision making

Comparaison de l'évaluation contingente et de l'analyse conjointe: une application à la mesure du consentement à payer pour un concept de service innovant (comparison of contingent valuation and conjoint analysis: an application to measure the willingness to pay for an innovative service concept)

Search strategy 3 (Scopus database): (population surveillance (keywords) AND cost-benefit analysis (Keywords) Search strategy 4 (Scopus database): TITLE(''Public health surveillance'') OR TITLE(watch)) AND TITLE(cost OR costs OR economic à OR financ à ) AND TITLE

The authors are very grateful to Ms. F. Biton for her help in the bibliographic search.Conflicts of interest: None declared.

Economic evaluations of public health surveillance systems have been little studied so far. This systematic review has retrieved only nine full economic evaluations regarding public health surveillance. In costbenefits analyses, benefits are estimated mainly by the costs of the health outcomes prevented. In cost-effectiveness analyses, different surveillance strategies are compared and benefits are estimated by the number of the health outcomes prevented by one strategy compared with another one. These approaches cannot take into account other positive non-monetary effects of public health surveillance. In the context of budgetary constraints, looking at efficiency of public health surveillance becomes increasingly necessary. There is a need in developing more research and studies in this field.