key: cord-0786414-7lho7hwb
authors: Claudia, Ebm; Fabio, Carfagna; Sarah, Edwards; Mantovani, Alberto; Maurizio, Cecconi
title: Potential harm caused by physicians' a-priori beliefs in the clinical effectiveness of hydroxychloroquine and its impact on clinical and economic outcome – A simulation approach
date: 2020-12-09
journal: J Crit Care
DOI: 10.1016/j.jcrc.2020.12.003
sha: 892c8874e5501cf7a36fce6646b801b0313e8fe3
doc_id: 786414
cord_uid: 7lho7hwb

Despite growing controversies around Hydroxychloroquine's effectiveness, the drug is still widely prescribed by clinicians to treat COVID19 patients. Therapeutic judgment under uncertainty and imperfect information may be influenced by personal preference, whereby individuals, to confirm a-priori beliefs, may propose drugs without knowing the clinical benefit. To estimate this disconnect between available evidence and prescribing behavior, we created a Bayesian model analyzing a-priori optimistic belief of physicians in Hydroxychloroquine's effectiveness. Methodology: We created a Bayesian model to simulate the impact of different a-priori beliefs related to Hydroxychloroquine's effectiveness on clinical and economic outcome. Results: Our hypothetical results indicate no significant difference in treatment effect (combined survival benefit and harm) up to a presumed drug's effectiveness level of 20%, with younger individuals being negatively affected by the treatment (RR 0.82, 0.55–1.2; (0.95 (1.1) % expected adverse events versus 0.05 (0.98) % expected death prevented). Simulated cost data indicate overall hospital cost (medicine, hospital stay, complication) of 18.361,41€ per hospitalized patient receiving Hydroxychloroquine treatment. Conclusion: Off-label use of Hydroxychloroquine needs a rational, objective and datadriven evaluation, as personal preferences may be flawed and cause harm to patients and to society.

The use of drugs that have not received regulatory approval, known as off-label prescribing, has been widely practiced before, [1] and continued to be applied during the COVID-19 crisis [2] . In the absence of proven medical therapies, the standard of care to treat viral pneumonia in patients with COVID-19, consists of supportive treatment aimed at allowing the body to rest and focus its energy on fighting the disease itself.

[3] Yet, when the standard of care (SOC) does not instantly translate into survival benefits, it is psychologically understandable that medical professionals want to trial new medical technology and treatments to help these vulnerable patients. [2, 4] This impulse to hope that the adjunct use of readily available medicines may translate into clinical benefits has led to boost several existing drugs to be used beyond their original indication. [5, 6] Such unsupported use of offlabel drugs has raised major concerns about safety and effectiveness. [1, 4] In fact, previous data showed that the majority of off-label drug use is limited or had no scientific support (73%); and was mainly based on personal preferences. [7, 8] One of the drugs, which has been proposed off-label to treat COVID-19 patients, is Hydroxychloroquine (HQC). Hydroxychloroquine, an anti-malaria drug, has known benefits for other infectious and autoimmune indications, but also has a worrying side effect profile ranging from retinopathy to life-threatening cardiac arrhythmia, prevalent in the general population including children and young patients. [9] [10] [11] At the beginning of the COVID-19 crisis, we witnessed a surge in non-randomized trials demonstrating some clinical benefits related to HQC for severe acute J o u r n a l P r e -p r o o f Journal Pre-proof respiratory syndrome coronavirus 2 (SARS-CoV) infected patients. However, the majority of these studies had severe limitations in the methodology, producing mainly controversial findings, with one large study even being retracted after publication due to data inconsistency. [12] [13] [14] Over the course of the following month several large randomized studies showed no benefit for patients exposed to HQC treatment. [15, 16] Despite rising evidence against its use, HQC is still being prescribed by clinicians and promoted by politicians, leading to its continued, yet unsupported off-label use. [17, 18] In the absence of an evidence-based, risk-benefit analysis conducted by relevant regulatory and scientific bodies, we need to be aware that a drug may benefit some patients, but can also expose them to unknown clinical harm, or be ineffective and hence cause waste in an already burdened healthcare system. [2, 19] As such, the off-label use of a drug without evidence of its effectiveness has an enormous economic impact, and, may waste valuable resources urgently needed in other areas. [20, 21] Such -allocative inefficacy‖ may distract resources away from more important priorities, for example, promising clinical research studies, increasing ICU bed capacity, augmenting ventilation support technology, among many other initiatives.

To quantify the gap between available evidence and personal preferences that may influence decisions, we estimated the impact of a-priori beliefs related to the effectiveness of Hydroxychloroquine on clinical and economic outcomes. We modeled different clinical scenarios, taking into account different stages of the disease manifestation as well as stratifying the patients according to different age and risk groups.

We designed a Bayesian hierarchical model to simulate the clinical effect and the overall costs of HCQ, when assuming different a-priori beliefs about the effectiveness of the drug. We chose the Bayesian approach, as it allowed for the incorporation of the uncertainty related to the available J o u r n a l P r e -p r o o f Journal Pre-proof knowledge through the specifications of prior distributions for all unknown parameters in the simulation model. [22, 23] Model Structure and Input Parameter: Our simulation pathway started with the entire population being potentially exposed to SARS-CoV (Figure 1) . After a positive COVID-19 test, the symptomatic patient received a course of HCQ treatment (10 days, 300-600mg twice a day). Based on the effectiveness of the drug, which we defined as a reduced probability of transiting from a mild to a severe clinical state (requiring intensive care admission), the subjects recovered or transited with predefined probability distributions of progression to the next node, conditional to the former node. In our example, we used effectiveness levels of 5%, 10%, 20%, 50% decrease with the probability of transiting into a severe state. To account for different risk profiles, we stratified the cohort according to age (I 0-9; II 10-19; III 20-29; IV 30-39; V 40-49; VI 50-59; VII 60-69, VIII 70-79 IX 80-89; X 90+). Prior specification on parameter distributions can be found in Table 1 (clinical probabilities) and Table 2 (cost input parameter). For example, if an elderly patient (70 years) has a 7% baseline risk of being SARS-CoV infected, once infected there is a 30% probability of transiting into a severe state with a 12% risk of death thereafter. In Table 2 , it can also be observed that the patient will occur additive costs along the pathway (hospitalization, complication, treatment costs, etc.), until death or discharge. [25, 26] Cost data were derived from literature research, and included direct costs such as treatment, hospital and ICU costs, human resources, minor and mayor complications, and indirect costs, such as loss of productivity. [27] [28] [29] For statistical reasoning, we followed international guidelines on conducting and

reporting Bayesian statistics. [30] Uncertainty: Because of the novelty of this illness and the many unknown factors related to its diffusion and mortality, we had to express a certain degree of uncertainty in our parameters. We used a Gibbs sampling, a Markov Chain Monte Carlo algorithm, to generate a sequence of samples from our set of input variables.

J o u r n a l P r e -p r o o f

We simulated a hypothetical population of 10^ {6} people, having the same age distribution as that of the Italian population (Figure 2) . A snapshot of the relative risk of death in severe ICU cases (posterior outcomes) of the estimations are presented in Figure 3 . These data illustrate that, if we prospectively predict that the drug has no or little effect on disease progression, this would cause harm to the population. Only if there is an a-priori believe that the drug can improve clinical progression by more than 20%, we may see a potential clinical benefit (Relative Risk (RR) 0.81, Confidence Interval (95% CI) 0.76-0.87) of the drug on survival, but only in the elderly population (60 years and above).

We further compared the expected side effects with the survival and determined the cut-off point, (the balance between severe side effects related of the drug and increased patient survival rate) to be at an effectiveness level slightly below 20%. At a presumed effectiveness level of 20%, we saw 266 patients surviving (ICU and hospital), while 336 patients experiencing severe side effects (Table 3) . Direct outcomes -Disease progression and ICU admission: In the general population, there is a severe disease progression of 41% (SD 5.1%) at a presumed drug effectiveness (pEff of 5%), 39%

(SD 4.8%) (pEff 10%), 35% (SD 4.5%) (pEff 20%) and 23% (SD 3.0%), compared to 43% (SD 5.5%) when no drug was given, with avoided ICU admission, ranging from 0.33% (SD 1.1%) (pEff 5%), 0.63% (SD 1%) (pEff 10%), 1% (SD 1.2%) (pEff 20%) to 3% (SD 0.92%) (pEff 50%)

Indirect effect -Survival: Figure 4 shows the expected risk reduction at an effectiveness level of 20%. When patients are stratified by age, we see an improvement in combined clinical outcomes only for a presumed effectiveness level above 20%. RR 0.81, 0.76-0.87 for a population older than 60 years.

Looking at lower presumed effectiveness levels (< 10%), we see no benefits in prescribing The safety and effectiveness profile for the off-label use of Hydroxychloroquine in COVID-19

patients is controversial. [32] [33] [34] Despite all the uncertainty related to the effectiveness of the drug and potential harm, [18] we witnessed a disproportionate belief in the drugs, evidenced in the fact that Hydroxychloroquine was still being used after six month of evidence accumulated on its lack of effectiveness for COVID-19 patients. [17, 35] Applying a Bayesian approach, we prospectively quantified that up to an effectiveness threshold of 20%, the drug had no benefit to the general population, and may cause unnecessary waste in the system. In particular, in young patients where case fatality rate is low, as in patient <20 years, we estimated harm caused by Hydroxychloroquine at 0.95 (1.1) % expected side effects versus 0.05 (0.98) % expected death prevented. By simulating different predictive clinical scenarios, taking into account personal preferences or cognitive bias, these results indicate that the belief in the effectiveness of the off-label use for Hydroxychloroquine was vastly overrated. While our data were predictive on the basis of hypothetical data, over the course of the COVID-19 pandemic, we saw emerging data confirm the ambiguity of the effectiveness of the drug, with some patient populations being exposed to ineffective treatment and even worse to potential harm. [32, 36, 37] One of the first studies, Tang et al. reported no superiority in conversion rates of SARS-CoV-2 (viral clearance) related to the treatment with Hydroxychloroquine, while side effects were apparent in up to 30% of the study population. [38] This was further evidenced by a recent large randomized study (UK Recovery) confirming the signal towards higher mortality in the Hydroxychloroquine arm, and this study arm was stopped preliminary due to safety and ethics concern.

[15] Further study results from other randomized trials such as REMACAP or SOLIDARITY [16, 39] The results warrant further research and we are initiating follow-up studies to explore if increased awareness on our internal bias will directly translate into better prescribing in our clinical routine.

Simulation models are always simplifications of the real systems being analyzed. Furthermore, we cannot forecast the future with precision, but only evaluate the situation at a point in time. The COVID-19 crisis has dynamics that alter our model input parameter continuously. Hence, this model can be used as a guidance, but our results need further confirmation within ongoing randomized trials.

We did not quantify or include data on indirect costs, such as costs of demoralizing staff due to proposing unreliable research activities, opportunity costs of non-conducting controlled research activities (defined as the lost opportunity to allocate scarce resources to activities which yield a better outcome in terms of effect and costs), or loss related to taking away the freedom of physicians to J o u r n a l P r e -p r o o f Journal Pre-proof explore and potentially develop new solutions. However, we stressed the importance that solid evidence is needed before advising on the use of any drug.

Finally, we stratified our risk groups according to age, and did not include comorbidities in our model.

At the time of modeling, no sufficient data were available on comorbidities and to avoid including more uncertainty we focused on age as a risk parameter.

Off-label use of Hydroxychloroquine needs a rational, objective and data-driven evaluation, as personal preferences may be flawed and cause harm to patients. Our Bayesian simulation highlights the vulnerability of a-priori beliefs of physicians prescribing off-label drugs, and its negative impact on clinical and economic outcomes. These data may be used to create awareness around biased preferences and may inform educational programs on statistical literacy for prescribing clinicians.

Transparency declarationthe lead author Prof. M Cecconi affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained. The model simulates the pathway of a cohort exposed to the SARS COV-2, receiving a therapeutic course of Hydroxychloroquine once exposed to COVID-19 

companies and regulatory authorities

Compassionate use of experimental drugs in the Ebola outbreak

Integrating Clinical Research into Epidemic Response: The Ebola Experience

Off-label treatments were not consistently better or worse than approved drug treatments in randomized trials

Off-label medication use in adult critical care patients

Side Effects of and Compliance with Malaria Prophylaxis in Children

Drug Evaluation during the Covid-19 Pandemic

QT prolongation, torsades de pointes, and sudden death with short courses of chloroquine or hydroxychloroquine as used in COVID-19: A systematic review

Retraction--Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis‖

Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial

Efficacy of hydroxychloroquine in patients with COVID-19: results of a randomized clinical trial

Effect of Hydroxychloroquine in Hospitalized Patients with COVID-19: Preliminary results from a multi-centre, randomized, controlled trial

Repurposed antiviral drugs for COVID-19 -interim WHO SOLIDARITY trial results

The Risks of Prescribing Hydroxychloroquine for Treatment of COVID-19-First, Do No Harm

Review of Current Evidence of Hydroxychloroquine in Pharmacotherapy of COVID-19

Optimizing the Trade-off Between Learning and Doing in a Pandemic

Health system efficiency: How to make measurement matter for policy and management

Health system efficiency: How to make measurement matter for policy and management

European Observatory on Health Systems and Policies

Use of Bayesian statistics in drug development: Advantages and challenges

Bayesian clinical trials in action

Integrated surveillance of COVID-19 in Italy

Baseline Characteristics and Outcomes of 1591 Patients Infected With SARS-CoV-2 Admitted to ICUs of the Lombardy Region, Italy

Johns Hopkins Hospital and Medicine. Coronavirus COVID-19 Global Cases by the Center for Systems Science and Engineering

Effect of costing methods on unit cost of hospital medical services

Reference Costs 2015-16

/17 National Tariff Payment System

The JASP Guidelines for Conducting and Reporting a Bayesian Analysis

Hydroxychloroquine for the management of COVID-19: Hope or Hype? A Systematic review of the current evidence

FDA cautions against use of hydroxychloroquine or chloroquine for COVID-19 outside of the hospital setting or a clinical trial due to risk of heart rhythm problems

Drug Evaluation during the Covid-19 Pandemic

Characterization and Clinical Course of 1000 Patients with COVID-19 in New York: retrospective case series

Effect of Hydroxychloroquine in Hospitalized Patients with Covid-19

Outcomes of hydroxychloroquine usage in United States veterans hospitalized with Covid-19

Hydroxychloroquine in patients with J o u r n a l P r e -p r o o f COVID-19: an open-label, randomized

The Randomized Embedded Multifactorial Adaptive Platform for Community-acquired Pneumonia (REMAP-CAP) Study: Rationale and Design