key: cord-1022180-yqwp2fta authors: Deng, Jiawen; Zhou, Fangwen; Hou, Wenteng; Heybati, Kiyan; Ali, Saif; Chang, Oswin; Silver, Zachary; Dhivagaran, Thanansayan; Ramaraju, Harikrishnaa Ba; Wong, Chi Yi; Zuo, Qi Kang; Lapshina, Elizabeth; Mellett, Madeline title: Efficacy of lopinavir–ritonavir combination therapy for the treatment of hospitalized COVID-19 patients: a meta-analysis date: 2022-01-31 journal: Future virology DOI: 10.2217/fvl-2021-0066 sha: fdc2971bdd9c72761dbefbd05a8660dbe430f25c doc_id: 1022180 cord_uid: yqwp2fta Aim: To evaluate the efficacy and safety of lopinavir–ritonavir (LPV/r) therapy in treating hospitalized COVID-19 patients. Materials & methods: Data from randomized and observational studies were included in meta-analyses. Primary outcomes were length of stay, time for SARS-CoV-2 test conversion, mortality, incidence of mechanical ventilation, time to body temperature normalization and incidence of adverse events. Results: Twenty-four studies (n = 10,718) were included. LPV/r demonstrated no significant benefit over the control groups in all efficacy outcomes. The use of LPV/r was associated with a significant increase in the odds of adverse events. Conclusion: Given the lack of efficacy and increased incidence of adverse events, the clinical use of LPV/r in hospitalized COVID-19 patients is not recommended. viral load in vitro [20] , as well as encouraging in vivo evidence suggesting that LPV/r may have been effective against the severe acute respiratory syndrome (SARS) with low incidences of adverse events in 2004 [21] [22] [23] . Both lopinavir and ritonavir are competitive inhibitors of viral proteases which prevents the post-translational proteolysis of precursor peptides and the subsequent release of functional viral proteins, resulting in the production of immature viral particles [24, 25] . They are commonly used in combination due to the low oral bioavailability of lopinavir, which requires ritonavir as a booster to achieve therapeutic drug concentration [26, 27] . It is postulated that LPV/r may bind to the highly conserved substrate-binding pocket region of the 3C-like proteinase (3CL pro ) of coronaviruses, leading to its anti-coronavirus capabilities [28, 29] . In silico binding studies involving LPV/r showed that LPV/r is capable of binding to the SARS coronavirus (SARS-CoV) proteases, and it is speculated that LPV/r should be able to bind to the 3CL pro protein of SARS-CoV-2 as well due to the highly conserved nature of the 3CL pro substrate binding site between SARS-CoV and SARS-CoV-2 [30, 31] . Current evidence regarding the clinical use of LPV/r is conflicting and of low quality. Several open-labeled international randomized controlled trials (RCTs), including the RECOVERY trial [32] , the WHO SOLIDARITY trial [33] and an RCT reported by Cao et al. [34] , have all reported a lack of benefits associated with LPV/r treatments in regards to mortality, viral clearance and time to clinical improvements. However, non-randomized observational studies [35] [36] [37] [38] [39] have often produced more optimistic results, suggesting that LPV/r may reduce time to viral clearance and viral shedding. The use of LPV/r for treating COVID-19 is still recommended in several countries, including China (in combination with interferon) [40] , Egypt, Saudi Arabia, Belgium and Ireland [41] , suggesting that the potential efficacy of LPV/r is still recognized, or at least, debated, by national health organizations. Conflicting opinions on the treatment of COVID-19 from medical guidelines could result in public mistrust, further exacerbating potential issues such as vaccine hesitancy [42, 43] . Previous systematic reviews and meta-analyses regarding the LPV/r usage for treating COVID-19 patients yielded no statistically significant difference between LPV/r and standard of care in multiple patient-important outcomes, such as mortality, disease progression and length of stay [44, 45] . However, these early reviews often included a small number of trials with low sample sizes, which may not be able to provide the necessary precision to detect significant treatment effects [46] . Most evidence-based guidelines were also based upon major RCTs such as the RECOVERY and SOLIDARITY trials, while not accounting for observational evidence. Lastly, many systematic reviews involving COVID-19 failed to account for data from non-English databases, namely Chinese databases, which may contain many unanalyzed trials due to the large number of COVID-19 cases in China during the early stages of the pandemic [47, 48] . Given the limitations of previous knowledge synthesis studies, we conducted an updated systematic review and meta-analysis to investigate whether the use of LPV/r, with or without adjuvant therapies, is more beneficial compared with standard of care or adjuvant therapies alone in regards to length of stay, time for positive-to-negative conversion of SARS-CoV-2 nucleic acid tests and mortality in hospitalized patients with COVID-19. We also examined the safety of LPV/r therapy in COVID-19 patients, including incidences of adverse events and severe (grade 3/4) adverse events. We conducted this systematic review and meta-analysis following recommendations from the Cochrane Handbook for Systematic Reviews of Interventions [49] and in accordance to the PRISMA statements [50] . See online Supplementary Table 1 for the completed PRISMA checklist. This review was prospectively registered on PROSPERO (CRD42021241183), the international prospective register of systematic reviews [51] . We searched the following databases from 1 January 2020 to 10 February 2021 using English search strategies consisting of the keywords 'lopinavir', 'ritonavir', 'lopinavir-ritonavir', 'LPV', 'Norvir' and 'Kaletra' in combination with database-specific COVID-19 search strings provided by the Rudolph Matas Library of the Health Sciences of Tulane University [52] : Medical Literature Analysis and Retrieval System Online (MEDLINE), Excerpta Medica Database (EMBASE) and PubMed. Additionally, we systematically searched the following Chinese databases from 1 January 2020 to 10 February 2021 using a custom Chinese search strategy: Wanfang Data, Wanfang Med Online, SinoMed, China National Knowledge Infrastructure (CNKI) and Chongqing VIP Information (CQVIP). The search strategies used for the database searches can be found in online Supplementary Tables 2-9. We did not impose language restrictions during our study identification and selection processes. Due to potential biased or problematic COVID-19 studies being published on preprint repositories [53] [54] [55] , we opted to limit our literature search to peer-reviewed sources only. We also hand-searched the reference sections of two previous systematic reviews [44, 45] for relevant studies that were not identified by our database searches. We included both RCTs and comparative non-randomized observational studies that satisfied the following inclusion criteria in our analysis: compared LPV/r with standard of care, or compared LPV/r with adjuvant therapies to adjuvant therapies alone, included laboratory-confirmed, hospitalized COVID-19 patients. While we included studies involving LPV/r with adjuvant therapies, we only included studies that used the same concurrent therapy for its intervention and control arms to minimize the effect of adjuvant therapies on treatment outcomes, similar to the design of several other meta-analyses [56] [57] [58] . Our primary outcomes include: length of stay, time for positive-to-negative SARS-CoV-2 nucleic acid tests, mortality at the latest follow-up, incidence of mechanical ventilation, time to normalization of body temperature and incidence of adverse events. Our secondary outcomes include: rate of positive-to-negative conversion at day 7 and day 14, and incidence of severe (grade 3/4) adverse events. We performed title and abstract screening using Rayyan (https://rayyan.qcri.org/) [59] independently and in duplicate based on the eligibility criteria. Included abstracts were entered into an independent and in duplicate full text screening process. We resolved disagreements by recruiting a senior author to attain consensus. Figure 1 shows the PRISMA flowchart [60] of our study selection process. We performed data extraction independently and in duplicate using extraction sheets developed a priori. We extracted information relating to baseline demographics, descriptions of study methodology, treatment descriptions and outcome measures. The full list of extracted items can be found on our PROSPERO registration. We assessed the risk of bias in the included RCTs using the revised Cochrane risk of bias tool for randomized trials (RoB2) [61] . The risk of bias in non-randomized observational studies was assessed using the risk of bias in non-randomized studies of interventions (ROBINS-I) [62] . Reviewers judged the risk of bias in each RoB2 or ROBINS-I domain independently and in duplicate, and resolved disagreements by consulting with a senior author. We assessed the quality of evidence for our primary outcomes using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) framework [63, 64] . The GRADE approach evaluates the quality of evidence by assessing the following domains: study limitations (risk of bias) [65] , indirectness [66] , inconsistency [67] , imprecision [68] and publication bias [69] . Quality of evidence may also be rated up due to magnitude of effects, dose-response gradients and plausible confounders [70] . The overall quality of evidence for each outcome was rated as either high, moderate, low or very low [71] , and the results of the primary outcomes and GRADE ratings were presented in a GRADE summary of findings table [72] generated using the GRADEpro online software (https://gradepro.org/) [73] . We conducted all statistical analyses using R 4.0.4 (https://www.r-project.org/) [74] , and we performed randomeffects meta-analyses using the meta 4.18 library (https://cran.r-project.org/web/packages/meta/) [75] . We expressed and pooled the treatment effects of dichotomous outcomes as odds ratios (ORs) and 95% confidence intervals (CIs), and we also calculated the number needed to treat (NNT) and harm (NNH) [76] for dichotomous outcomes. We reported the treatment effect of continuous outcomes as mean differences (MDs) and 95% CIs. For studies with missing data required for analysis, including the mean outcome value and measure of variance for continuous outcomes, attempts were made to contact corresponding authors to obtain unpublished data. For studies that presented continuous outcomes in median and interquartile range (IQR), we assumed that the outcome data is normally distributed and used methods recommended by Luo et al. [77] and Wan et al. [78] to estimate the mean and standard deviation (SD) for analysis. We tested the impact of this assumption by conducting a subgroup analysis comparing the pooled results from studies with estimated mean and SD to studies that did not require estimation. For studies reporting zero events in one of its treatment arms, we applied a continuity correction factor of 0.5 [79] to complete the meta-analysis. We did not include studies that reported zero events in all treatment arms in our analyses. We assessed the presence of heterogeneity using the Cochran's Q test [80] with a significance level of p < 0.10, as recommended by the Cochrane Handbook [49] . Heterogeneity was quantified using I 2 statistics [80, 81] . We interpreted 30% < I 2 < 75% as moderate heterogeneity and I 2 ≥ 75% as serious heterogeneity [49] . We drew funnel plots [49] to identify small study effects within our included studies as a signal for the presence of publication bias, and we used Egger's regression test to quantitatively evaluate asymmetry within the funnel plots [82] . Egger's regression test was not conducted if fewer than 10 studies were included in the analysis, as the test may lack power in these circumstances [83] . We used the trim-and-fill method [84, 85] to estimate the number of missing, unpublished studies and to observe the impact of unpublished studies on the pooled treatment effect when potential publication bias is detected. We performed meta-regression analyses on the proportion of patients with severe disease, defined according to individual study criteria. We performed subgroup analyses based on factors defined a priori: study design (randomized vs nonrandomized), adjuvant regimens and daily LPV/r dosage. For the outcome of mortality, we also conducted a meta-regression to examine the impact of different follow-up durations on the treatment effect. We planned to conduct subgroup analyses based on risk of bias rating (low/some concerns RoB2 + low/moderate ROBINS-I vs high RoB2 + serious/critical ROBINS-I). However, this was not completed as only observational studies were included in the high risk subgroup, and only RCTs were included in the low risk subgroup. This observation makes the subgroup analysis by risk of bias rating redundant. We identified and screened 2030 (after deduplication using Endnote 20 [https://endnote.com/]) potentially eligible titles/abstracts. One hundred and twenty four full texts were retrieved and screened. Four RCTs [32] [33] [34] 86] and 20 observational studies [36] [37] [38] [87] [88] [89] [90] [91] [92] [93] [94] [95] [96] [97] [98] [99] [100] [101] [102] [103] were ultimately included in the systematic review and meta-analysis ( Figure 1 ) with a total of 10,718 hospitalized COVID-19 patients. Characteristics of included studies are tabulated in Table 1 . Four studies [89] [90] [91] 95] only included patients hospitalized in intensive care units (ICUs), while Yao et al. [100] excluded ICU-hospitalized patients. Cao et al. [34] and Lecronier et al. [91] only included patients with severe disease, while four studies [86, 88, 92, 98] excluded severe patients. The risk of bias in the included RCTs was assessed using RoB2 ( Figure 2A ). Three RCTs [32] [33] [34] were rated as having some concerns for risk of bias due to their unblinded, open-label design. The remaining RCT by Li et al. [86] was rated as having a low risk of bias. The risk of bias in included observational studies was assessed using ROBINS-I ( Figure 2B ). Ten studies [36, 87, 88, [93] [94] [95] 97, 98, 100, 103] were rated as having a serious risk of bias, while ten studies [37, 38, [89] [90] [91] [92] 96, 99, 101, 102] were rated as having a critical risk of bias. Five studies [36, 38, 89, 90, 103] were rated as having a serious risk of bias due to confounding factors, while one study [37] was rated as having a critical risk of bias due to either a lack of reporting for important patient characteristics (e.g., disease severity) or an imbalance in the reported patient characteristics. In terms of risk of bias due to selection of participants, three studies [38, 92, 103] were rated as having a serious risk of bias due to suspected selection bias, such as only including discharged patients in retrospective analyses. Five studies [37, 38, 90, 92, 96] were rated as having a serious risk of bias due to classification of interventions, mainly due to poorly defined inclusion criteria (e.g. studies that included patients who had received any LPV/r treatment during hospitalization without describing the LPV/r regimen). Two studies [37, 96] were rated as having a serious risk of bias due to deviations from the intended interventions due to imbalances in the adjuvant therapy received between treatment arms. Lastly, two studies [101, 102] were rated as having a serious risk of bias due to missing patient outcome data. Risk of bias due to selection of reported results cannot be evaluated for all observational studies because none of the studies provided their respective study protocols. A total of 17 studies [32] [33] [34] 37, 38, 86, [88] [89] [90] [91] [92] [93] [94] [95] [96] 101, 103] (4 RCTs and 13 observational studies) with 10,105 patients examined the effect of LPV/r on mortality in hospitalized COVID-19 patients. Three studies [37, 86, 88] reported 0 death in all treatment arms and were excluded from the meta-analysis. The pooled OR was 0.77 (95% CI: 0.45-1.30) with significant and moderate heterogeneity (P Q <0.01, I 2 = 64%; Figure 3 ). The NNT was 26.2 patients. There were no significant differences between the pooled ORs from any subgroups, although heterogeneity was substantially reduced for the pooled treatment effect from RCTs only (I 2 = 0%; P Q = 0.54; Supplementary Figure 1 ), studies utilizing a daily regimen of LPV/r 0.8 g (I 2 = 25%; P Q = 0.25; Supplementary Figure 2) , and studies utilizing hydroxychloroquine as an adjuvant therapy (I 2 = 0%; P Q = 0.43; Supplementary Figure 3 ). There were no significant correlations between the proportion of patients with severe disease and the treatment effect based on the meta-regression analysis from six studies [34, 91, 92, 96, 101, 103] (p = 0.69; Supplementary Figure 4 ). Additionally, there were also no significant correlations between the follow-up duration and the treatment effect based on meta-regression of six studies [32, 34, 91, [93] [94] [95] (p = 0.20; Supplementary Figure 5 ). There was no publication bias based on visual inspection of the funnel plot. This is corroborated by Egger's regression test, which did not detect any significant small study effect (P Egger = 0.22; Supplementary Figure 6 ). A total of nine studies [32, 34, 37, 38, 92, 94, 96, 100, 103] (2 RCTs and 7 observational studies) with 6537 patients examined the effect of LPV/r on length of stay. An observational study conducted by Nathalie et al. [92] was included in the analysis as two separate entries, as the length of stay was reported separately for patients taking hydroxychloroquine adjuvants versus patients who were not taking additional adjuvants beyond standard of care. The pooled MD was 1.56 (95% CI: -0.70-3.82) with significant and severe heterogeneity (P Q < 0.01; I 2 = 85%; Figure 4 ). 10 There were significant between-group differences for the subgroup analysis by different adjuvant therapies, as a study conducted by Wang et al. [96] Figures 7-9) , although heterogeneity was significantly reduced in the RCT subgroup (P Q = 0.58; I 2 = 0%; Supplementary Figure 10 ). Meta-regression analysis based on six studies [34, 37, 92, 96, 100, 103] showed no significant correlations between the proportion of patients with severe disease and the treatment effect (p = 0.61; Supplementary Figure 11 ). There was no publication bias based on visual inspection of the funnel plot (Supplementary Figure 12) . A total of nine studies [36] [37] [38] 86, 88, 94, 97, 99, 101] (1 RCTs and 8 observational studies) with 914 patients examined the effect of LPV/r on time for positive-to-negative conversion of SARS-CoV-2 nucleic acid tests. The pooled MD was -1.87 (95% CI: -4.00-0.26) with significant and severe heterogeneity (I 2 = 76%; P Q < 0.01; Figure 5A ). There were significant between-group differences for the subgroup analysis by different adjuvant therapies (Supplementary Figure 13) , as a study conducted by Panagopoulos et al. [38] that used hydroxychloroquine and azithromycin as adjuvants reported a MD of -4.94 (95% CI: -7.13 to -2.75), which is substantially lower compared with the MD reported by Liu et al. [97] (MD: 1.73 [95% CI: -2.38-5.83]) which used IFN therapy as an adjuvant, as well as the pooled MD of studies that did not use adjuvants beyond standard of care (MD: -1.81 [95% CI: -4.16-0.55]). We did not perform the subgroup analysis by LPV/r regimens, as all studies included in the analysis reported using the same regimen of LPV/r 1.0 g q.d., with the exception of Panagopoulos et al. [38] which did not report the LPV/r regimens used in the study. There were no significant differences in the MDs between studies of different methodological designs, nor between studies that required imputation versus studies that did not require imputation (Supplementary Figures 14 & 15) . 10 Meta-regression analysis based on 5 studies [37, 86, 88, 99, 101] showed no significant correlations between the proportion of patients with severe disease and the treatment effect (p = 0.10; Supplementary Figure 16 ). There was no publication bias based on visual inspection of the funnel plot (Supplementary Figure 17) . At 7 days after the commencement of LPV/r therapy, the OR of negative SARS-CoV-2 tests in the LPV/r group compared with the control group was 0.85 (95% CI: 0.44-1.65; NNH 25.4 patients) with no significant heterogeneity (I 2 = 20%; P Q = 0.28; Figure 5B ) based on six studies [86, 87, 95, 98, 102, 103] . At 14 days, the OR of negative SARS-CoV-2 tests was 1.03 (95% CI: 0.34-3.16, NNT 144.0 patients) with no significant heterogeneity (I 2 = 36%; P Q = 0.19, Figure 5C ) based on four studies [34, 86, 87, 91] . There were no significant differences between the pooled ORs from any subgroups for the aforementioned secondary outcomes (Supplementary Figures 18-22) , nor were there any significant correlations between the proportion of patients with severe disease and the treatment effect based on meta-regression analyses (Supplementary Figures 23 & 24 ). There were no detectable small study effects in any of the aforementioned secondary outcomes based on visual inspections of the funnel plots ( Figures 25 & 26) . Nine studies [17, 33, 34, 38, [89] [90] [91] 94, 103] (3 RCTs and 6 observational studies) with 8240 patients assessed the effect of LPV/r on incidences of mechanical ventilation in hospitalized COVID-19 patients. The pooled OR was 1.04 (95% CI: 0.55-1.97) with no significant heterogeneity (I 2 = 32%; P Q = 0.16; Figure 6 ). The NNH was 263.3 patients. There were no significant between-group differences in any of the subgroup analyses (Supplementary Figures 27-29) . We did not perform meta-regression analyses due to insufficient data. No evidence of small study effects was observed based on visual inspections of the funnel plot (Supplementary Figure 30) . Five studies [36, 38, 88, 99, 100] (all retrospective cohort studies) with 313 patients assessed the effect of LPV/r on time to body temperature normalization in hospitalized COVID-19 patients. The pooled MD was -0.04 (95% CI: -2.34-2.25) with significant and substantial heterogeneity (I 2 = 82%; P Q < 0.01; Figure 7 ). There were no significant between-group differences in any of the subgroup analyses (Supplementary Figures 31-32) . Meta-regression of outcomes from three studies [88, 99, 100] did not identify a significant correlation between the proportion of patients with severe disease and the treatment effect (p = 0.20; Supplementary Figure 33 ). We did not detect evidence of small study effects based on visual inspections of the funnel plot (Supplementary Figure 34) . A total of six studies [34, 86, 87, 99, 101, 102] 1.04-7.95) with significant but moderate heterogeneity (I 2 = 67%; P Q < 0.01; Figure 8A ). The NNH was 4.6 patients. The most commonly reported adverse events in the LPV/r group were gastrointestinal side effects such as diarrhea, nausea, vomiting, stomach pain and loss of appetite [34, 87, 89, 99, [101] [102] [103] and liver-related toxicities such as elevated liver enzymes and elevated bilirubin [89, 101, 103] . Subgroup analysis by different LPV/r regimens was not conducted as all studies included in the analysis reported using the same regimen of LPV/r 1.0 g q.d. There were no significant between-group differences in any of the subgroup analyses ( Supplementary Figures 35 & 36) . Meta-regression analysis of 6 studies [34, 86, 87, 99, 101, 102] showed no significant correlations between the proportion of patients with severe disease and the treatment effect (p = 0.11; Supplementary Figure 37 ). Visual inspection of the funnel plot showed no significant small study effects (Supplementary Figure 38) . 5 patients need to be treated with LPV/r to cause one additional incidence of adverse events † The risk in the intervention group (and its 95% Cl) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). ‡ Downgraded due to study limitations; a majority of included studies were rated as having serious or critical risk of bias according to ROBINS-I. § Downgraded due to inconsistency; significant and severe heterogeneity was observed in the analysis. ¶ Downgraded due to imprecision; confidence intervals could not rule out the possibility of no effect (crosses null). # Upgraded due to a large magnitude of effect. GRADE Working Group quality of evidence rating [71] . High quality: We are very confident that the true effect lies close to that of the estimate of the effect. Moderate quality: We are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low quality: Our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect. Very low quality: We have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect. GRADE: Grading of Recommendation, Assessment, Development and Evaluation; LPV/r: Lopinavir-ritonavir; MD: Mean difference; OR: Odds ratio; OS: Observational study; RCT: Randomized controlled trial. Two RCTs by Cao et al. [34] and Li et al. [86] , which both used a regimen of LPV/r 1.0 g q.d. with no adjuvant therapy beyond standard of care, reported incidences of severe adverse events. The pooled OR was 2.10 (95% CI: 0. 85-5.20) , with no significant heterogeneity (I 2 = 0%; P Q = 0.86; Figure 8B ). The NNH was 11.7 patients. The summary of findings for primary outcomes is tabulated in Table 2 . Our systematic review and meta-analysis included 4 RCTs and 20 observational studies involving 10,718 hospitalized COVID-19 patients. On the basis of very low quality of evidence, we found that LPV/r use did not significantly decrease the odds of death, the length of stay, the time for positive-to-negative conversion of SARS-CoV-2 nucleic acid tests, or the time to normalization of body temperature. In fact, the use of LPV/r was associated with a non-significant increase in length of stay compared with standard of care or adjuvant therapies alone. We also did not find any benefits associated with LPV/r in reducing incidences of mechanical ventilation, based on a low quality of evidence. The lack of efficacy in LPV/r use as shown by our primary outcomes is supported by results from our secondary outcomes as well, since we did not identify a significant increase in the incidence of positive-to-negative conversions of SARS-CoV-2 nucleic acid tests in the LPV/r group compared with the control group at 7 or 14 days after the start of treatment. In general, results of our subgroup analyses and meta-regressions did not yield any significant findings either, with the exception of two single-study subgroups: Wang et al. [96] , which found that the use of an IFN adjuvant therapy with LPV/r significantly increased the length of stay compared with IFN therapy alone, and Panagopoulos et al. [38] , which found that the use of hydroxychloroquine + azithromycin in combination with LPV/r resulted in significant reductions in the time for positive-to-negative conversion of SARS-CoV-2 nucleic acid tests compared with only hydroxychloroquine + azithromycin. However, as no data pooling was conducted for these subgroups, it is unknown whether these statistically significant results are due to subgroup factors or confounding variables unique to each study. Notably, Panagopoulos et al. [38] only included 16 patients in their study, which is the lowest sample size among all of our included publications. Previous investigations have found that studies with small sample sizes tend to overestimate treatment effects [104, 105] ; thus, the significant findings from Panagopoulos et al. [38] may be due to an exaggeration of the treatment effect away from the null and should be interpreted within the context of other similar studies. Nathalie et al. [92] and Echarte-Morales et al. [93] both used hydroxychloroquine as an adjuvant therapy with large sample sizes, and they did not find any significant benefits associated with using LPV/r in combination with hydroxychloroquine (with or without azithromycin) in other primary outcomes, including length of stay and death. This suggests that the results from single-study subgroups with low sample sizes should be interpreted with caution and require further examination in future investigations. In addition to a lack of efficacy of LPV/r in the treatment of hospitalized COVID-19 patients, our meta-analysis also suggested that LPV/r may result in a significant 180% increase in the odds of adverse events based on moderate heterogeneity and moderate quality of evidence. While we did not specifically analyze the incidences of different categories of adverse events, gastrointestinal side effects and liver-related toxicities were the most commonly reported types of adverse events in the LPV/r group. The lack of efficacy of LPV/r in COVID-19 patients is consistent with previous systematic reviews, which found that LPV/r did not significantly reduce time to virological cure, time to body temperature normalization, or incidences of cough relief [44] . LPV/r also did not decrease incidences of disease progression or mortality [45] . LPV/r was considered to be a potential treatment regimen for SARS-CoV-2 due to both in vitro and in vivo studies which demonstrated that it may be effective in inhibiting the action of other human coronaviruses, including SARS-CoV and MERS-CoV [21, 23, 106, 107] . However, the most widely cited in vivo study conducted by Chu et al. [21] in SARS-CoV patients, which showed that LPV/r treatments may lower the severity of SARS-CoV symptoms, was an open, non-randomized trial involving a small treatment group and historical controls. The nature of the study design may be subjected to bias, and the results may be exaggerated due to improvements in disease management during the later stages of the 2004 epidemic. In addition, many in vitro studies yielded different IC 50 values in regards to the inhibitory effect of lopinavir on human coronaviruses, from 50 μmol/l in SARS-CoV [107] to 8-12 μmol/l in SARS-CoV-2 [108] . A study conducted by Wu et al. [107] attempted to synthesize lopinavir derivatives in an effort to improve the IC 50 of lopinavir against SARS-CoV. However, they were only able to obtain a minimum IC 50 of 25 μmol/l [107] . These values are generally greater than the maximum serum concentration (C max ) of lopinavir, which has been estimated to be 13.5 μmol/l [109] , and significantly greater than the unbound C max of lopinavir, which has been estimated to be 0.2 μmol/l [110] . The IC 50 values of lopinavir against coronaviruses are thousands of folds greater than the IC 50 values required to inhibit HIV, which has been estimated to be around 0.004-0.011 μmol/l for lopinavir [108, 111] . Given these findings, it has been suggested that the serum concentration of free LPV/r cannot suppress the action of SARS-CoV-2 in vivo, despite optimistic in vitro findings. This conclusion is supported by our current meta-analysis, which found that LPV/r lacks efficacy in a clinical setting. Apart from the lack of efficacy, previous reviews have also identified an increased incidence of adverse events in the LPV/r group compared with standard of care [44] , similar to our findings. Specifically, one review found that the use of LPV/r is significantly associated with a higher incidence of diarrhea [45] , which we have also observed in five of our included studies [34, 87, 99, 101, 102] although we did not quantitatively analyze the incidence of diarrhea. In addition to gastrointestinal side effects, we also found hepatotoxicity to be widely reported across our included studies. Levy et al. [89] reported observing higher incidences of jaundice and bilirubin elevation in the LPV/r group compared with standard-of-care, Yu et al. [101] reported elevated transaminase in LPV/r patients, and J. Wang et al. [103] reported elevated aspartate aminotransferase and alanine aminotransferase. Gastrointestinal side effects and hepatotoxicity are both known adverse events associated with the use of LPV/r [112] , with some studies reporting a prevalence of liver toxicities as high as 10% in HIV patients using LPV/r [113, 114] . Levy et al. [89] noted that hepatotoxicity associated with LPV/r may be particularly problematic for the treatment of COVID-19 patients with obesity, as underlying liver steatosis may be a contributing factor to liver toxicity. Recent reports have also raised concerns with regards to potential drug-drug interactions associated with LPV/r in the treatment of COVID-19 [115] . When used as a monotherapy, lopinavir has poor oral bioavailability because it is rapidly metabolized by cytochrome P450 enzyme systems [116] . For this reason, ritonavir is often co-formulated with lopinavir to take advantage of ritonavir's ability to potently inactivate cytochrome P4503A4 (CYP3A4), thus dramatically increasing the serum availability of lopinavir [117, 118] . However, inhibition of CYP3A4 may also increase the serum concentration of other drugs that rely on CYP3A4 for metabolism, potentially resulting in drug-drug interactions [119] . For example, systemic corticosteroids (i.e. dexamethasone), which have been recently recommended for use in COVID-19 patients receiving supplemental oxygen [120, 121] , relies on CYP3A4 for metabolism [122] . Thus, the co-administration of corticosteroids and LPV/r may result in Cushing's syndrome and adrenal suppression [123, 124] . Apart from COVID-19 treatments, medications prescribed for other conditions such as statins [125] and antiarrhythmic drugs [126] may also result in drug-drug interactions with LPV/r [115] . A previous observational study by Macías et al. [115] concluded that drug-drug interactions which contraindicates the use of LPV/r is often overlooked in the context of the ongoing COVID-19 healthcare crisis, signifying that physicians treating COVID-19 patients using LPV/r should be aware of potential interactions associated with CYP3A4 inhibition. While the guidance from the NIH [127] and IDSA [128] currently recommend against the use of LPV/r, these are primarily based upon major RCTs, namingly the RECOVERY and SOLIDARITY trial. The use of LPV/r for the management of COVID-19 patients continues to be recommended in the clinical guidelines across several major countries, including China [40] , Egypt, Saudi Arabia, Belgium and Ireland [41] . Our systematic review incorporated both randomized and observational evidence, and considered the impact of dosage on patient-important outcomes to provide comprehensive evidence on LPV/r's lack of efficacy. Additionally, given the significant number of adverse events associated with LPV/r, our findings could inform the perspectives of clinicians, policy makers and the general public to ensure patient safety. This systematic review and meta-analysis has several strengths. First, we performed several subgroup analyses examining the impact of our study methodologies on the treatment effect, such as comparing the pooled effect from studies that required imputation of the mean and SD with studies that did not require imputation. Secondly, we examined the effect of different LPV/r regimens and adjuvant therapies on the treatment outcomes using subgroup analyses. And lastly, we compared the results from studies with a low risk of bias (which were all RCTs, coincidentally) with the results from studies with a high risk of bias (which were all observational and non-randomized studies). Compared with previous meta-analyses, our systematic review included more studies involving a larger sample size of patients, which helped improve the power and precision of our analyses. This was possible, in part, due to additional database searches conducted in Chinese literature sources. Additionally, we evaluated the quality of our evidence using the GRADE framework, which was not done in previous reviews. However, our study also has several key limitations. Firstly, our subgroup analyses by different regimens and adjuvants often consist of many single-study subgroups, which severely limits the applicability of our subgroup analyses. Furthermore, a majority of our included studies are non-randomized observational studies, which are prone to biases and increased heterogeneity. According to ROBINS-I, all of our included observational studies were rated as having a serious or critical risk of bias, mainly due to potential confounding factors or inadequate descriptions of treatment regimes. Recent reports have shown that there has been a general decline in the quality of research articles and clinical trials [129, 130] during the COVID-19 pandemic due to the increased pace of publications, which may explain the poorer quality of the articles that we included. Nevertheless, the pooled results from our observational studies were consistent with the pooled results from high or moderate quality RCT studies according to our subgroup analyses, which increases the confidence in our findings. This systematic review and meta-analysis indicates, based on low to very low quality of evidence, that the use of LPV/r is not significantly associated with reductions in mortality, length of stay, time for positive-to-negative conversion of SARS-CoV-2 nucleic acid test, incidence of mechanical ventilation, or time to body temperature normalization in hospitalized patients with COVID-19. We also found that the use of LPV/r is significantly associated with an increase in adverse events, based on moderate quality of evidence. Considering the lack of efficacy and the risk of increased gastrointestinal and liver-related toxicities, as well as potential drug-drug interactions with other COVID-19 treatments, the use of LPV/r therapy for treating COVID-19 inpatients is not recommended based on the available evidence. • Lopinavir-ritonavir (LPV/r) combination therapy is an antiretroviral medication that was repurposed for the treatment of COVID-19 due to promising results from in vivo and in silico studies involving human coronaviruses. • In this systematic review and meta-analysis, results from 20 observational studies and 4 randomized controlled trials (n = 10,718) were included to examine the efficacy and safety of LPV/r for treating hospitalized COVID-19 patients. • A majority of the included RCTs were rated as having some concerns in regards to risk of bias using RoB2, while all of the included observational studies were rated as having serious or critical risk of bias using ROBINS-I. • The use of LPV/r was not associated with any significant benefit compared with standard of care or adjuvant therapies alone in reducing mortality, length of stay, time for positive-to-negative conversion of SARS-CoV-2 nucleic acid test, incidence of mechanical ventilation, or time to body temperature normalization. • The use of LPV/r was significantly associated with increased odds of adverse events (OR: 2.88; 95% CI: 1.04-7.95). The most common adverse events observed in the LPV/r group included gastrointestinal side effects and possible hepatotoxicity. • All efficacy outcomes were based on low to very low quality of evidence, while the outcome of adverse event incidence was based on moderate quality of evidence, according to the GRADE approach. • Due to the lack of efficacy and increased odds of adverse events, the clinical use of LPV/r for the treatment of hospitalized patients with COVID-19 is not recommended based on the available evidence. To view the supplementary data that accompany this paper please visit the journal website at: www.futuremedicine.com/doi/ suppl/10.2217/fvl-2021-0066 Author contributions J Deng was responsible for the conception and design of the work, as well as for supervising the data acquisition process (including search strategy development, database search, article screening, full-text retrieval, data extraction and risk of bias analyses), data analysis (including meta-analyses, meta-regressions, subgroup analyses and GRADE ratings) and drafting of the final manuscript. F Zhou was responsible for the conception and design of the work, as well as for supervising and contributing to all parts of the data acquisition process (including search strategy development, database search, article screening, full-text retrieval, data extraction and risk of bias analyses) and revised the final manuscript critically for important intellectual content. W Hou, K Heybati, S Ali, O Chang, Z Silver, T Dhivagaran, H Ba Ramaraju, CY Wong, QK Zuo, E Lapshina and M Mellett contributed to all parts of the data acquisition process (including article screening, data extraction, and risk of bias analyses) and revised the final manuscript critically for important intellectual content. The first and corresponding author had full access to all data in the study and is solely responsible for the decision to submit for publication. All authors had given final approval for the final version of the manuscript to be submitted for publication, and all authors agree to be held accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties. No writing assistance was utilized in the production of this manuscript. A comprehensive review of COVID-19 characteristics COVID-19 pathophysiology: a review WHO declares COVID-19 a pandemic COVID-19: WHO declares pandemic because of 'alarming levels' of spread, severity, and inaction Safety and efficacy of the BNT162b2 mRNA COVID-19 vaccine Safety and immunogenicity of ChAdOx1 nCoV-19 vaccine administered in a prime-boost regimen in young and old adults (COV002): a single-blind, randomised, controlled, phase 2/3 trial An mRNA vaccine against SARS-CoV-2 -preliminary report COVID-19: US cases surge but vaccine distribution is slow The challenges of distributing COVID-19 vaccinations US National Library of Medicine Therapeutic strategies for severe COVID-19: a position paper from the Italian Society of Infectious and Tropical Diseases (SIMIT) Therapy for early COVID-19: a critical need Hydroxychloroquine and COVID-19: a tale of populism and obscurantism Chloroquine and hydroxychloroquine in coronavirus disease 2019 (COVID-19). Facts, fiction and the hype: a critical appraisal Remdesivir for the treatment of COVID-19 -final report Dexamethasone in hospitalized patients with COVID-19 Effect of dexamethasone on days alive and ventilator-free in patients with moderate or severe acute respiratory distress syndrome and COVID-19: the CoDEX randomized clinical trial Lopinavir/ritonavir: repurposing an old drug for HIV infection in COVID-19 treatment In vitro activity of lopinavir/ritonavir and hydroxychloroquine against severe acute respiratory syndrome coronavirus 2 at concentrations achievable by usual doses Role of lopinavir/ritonavir in the treatment of SARS: initial virological and clinical findings Pharmacologic treatment of SARS: current knowledge and recommendations Screening of an FDA-approved compound library identifies four small-molecule inhibitors of Middle East respiratory syndrome coronavirus replication in cell culture Current progress in antiviral strategies ABC of AIDS. Antiretroviral drugs Population pharmacokinetics of lopinavir/ritonavir (Kaletra) in HIV-infected patients Insights into antiviral mechanisms of remdesivir, lopinavir/ritonavir and chloroquine/hydroxychloroquine affecting the new SARS-CoV-2 Design of wide-spectrum inhibitors targeting coronavirus main proteases Coronaviruses -drug discovery and therapeutic options Identification of potential binders of the main protease 3CL of the COVID-19 via structure-based ligand design and molecular modeling Old drugs as lead compounds for a new disease? Binding analysis of SARS coronavirus main proteinase with HIV, psychotic and parasite drugs Lopinavir-ritonavir in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial A landmark randomized controlled trial investigating the efficacy and safety of lopinavir-ritonavir for treating COVID-19 inpatients Repurposed Antiviral Drugs for COVID-19 -Interim WHO Solidarity Trial Results A landmark randomized controlled trial investigating the efficacy and safety of several repurposed antiviral medications for treating COVID-19 inpatients A trial of lopinavir-ritonavir in adults hospitalized with severe COVID-19 A landmark randomized controlled trial investigating the efficacy and safety of lopinavir-ritonavir for treating COVID-19 inpatients Efficacy and safety of Chinese herbal medicine versus lopinavir-ritonavir in adult patients with coronavirus disease 2019: a non-randomized controlled trial Clinical efficacy of lopinavir/ritonavir in the treatment of coronavirus disease 2019 Factors associated with prolonged viral shedding and impact of lopinavir/ritonavir treatment in hospitalised non-critically ill patients with SARS-CoV-2 infection Lopinavir/ritonavir as a third agent in the antiviral regimen for SARS-CoV-2 infection Lopinavir-ritonavir versus hydroxychloroquine for viral clearance and clinical improvement in patients with mild to moderate coronavirus disease 2019 National Health Commission of the People's Republic of China. Novel coronavirus pneumonia diagnosis and treatment plan (trial version 8) COVID-19 treatment guidelines: do they really reflect best medical practices to manage the pandemic? Public perceptions of conflicting information surrounding COVID-19: results from a nationally representative survey of U.S. adults Assessment of COVID-19 information overload among the general public Efficacy and safety of lopinavir/ritonavir for treatment of COVID-19: a systematic review and meta-analysis A previous systematic review assessing the efficacy and safety of lopinavir-ritonavir therapy for the treatment of COVID-19 Safety and efficacy of lopinavir/ritonavir combination in COVID-19: a systematic review, meta-analysis, and meta-regression analysis • A previous systematic review assessing the efficacy and safety of lopinavir-ritonavir therapy for the treatment of COVID-19 Sample size and the precision of the confidence interval in meta-analyses The prevalence of depression, anxiety, and sleep disturbances in COVID-19 patients: a meta-analysis The prevalence of depressive symptoms, anxiety symptoms and sleep disturbance in higher education students during the COVID-19 pandemic: a systematic review and meta-analysis Cochrane Handbook for Systematic Reviews of Interventions Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement PROSPERO: an international register of systematic review protocols Library guides: TU GOARN: COVID-19 search strategies Fast news or fake news?: the advantages and the pitfalls of rapid publication through pre-print servers during a pandemic COVID-19 and misinformation: how an infodemic fuelled the prominence of vitamin D The pandemic of publications: are we sacrificing quality for quantity? Pharmacological prevention of fractures in patients undergoing glucocorticoid therapies: a systematic review and network meta-analysis Efficacy of therapeutic plasma exchange for treatment of autoimmune hemolytic anemia: a systematic review and meta-analysis of randomized controlled trials Transfusion of modified blood components for the treatment of autoimmune hemolytic anemia: a network meta-analysis Rayyan -a web and mobile app for systematic reviews The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration RoB 2: a revised tool for assessing risk of bias in randomised trials ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions GRADE: an emerging consensus on rating quality of evidence and strength of recommendations Clinical practice guidelines: a primer on development and dissemination GRADE guidelines: 4. Rating the quality of evidence -study limitations (risk of bias) GRADE guidelines: 8. Rating the quality of evidence -indirectness GRADE guidelines: 7. Rating the quality of evidence -inconsistency GRADE guidelines 6. Rating the quality of evidence -imprecision GRADE guidelines: 5. Rating the quality of evidence -publication bias GRADE guidelines: 9. Rating up the quality of evidence GRADE guidelines: 3. Rating the quality of evidence GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables Using systematic reviews in guideline development: the GRADE approach Data analysis using R programming The statistical basis of meta-analysis Number needed to treat (or harm) Optimally estimating the sample mean from the sample size, median, mid-range, and/or mid-quartile range Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range The continuity correction Measuring inconsistency in meta-analyses Bias in meta-analysis detected by a simple, graphical test Evidence-based medicine and systematic reviews A nonparametric 'trim and fill' method of accounting for publication bias in meta-analysis Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis Efficacy and safety of lopinavir/ritonavir or arbidol in adult patients with mild/moderate COVID-19: an exploratory randomized controlled trial The efficacy and safety of lopinavir/ritonavir and arbidol in patients with coronavirus disease 2019 Brief report: retrospective evaluation on the efficacy of lopinavir/ritonavir and chloroquine to treat nonsevere COVID-19 patients Caution with the use of lopinavir/ritonavir in severely ill patients for the treatment of SARS-CoV-2: a report of severe jaundice Characteristics and outcomes of acute respiratory distress syndrome related to COVID-19 in Belgian and French intensive care units according to antiviral strategies: the COVADIS multicentre observational study Comparison of hydroxychloroquine, lopinavir/ritonavir, and standard of care in critically ill patients with SARS-CoV-2 pneumonia: an opportunistic retrospective analysis Early experimental COVID-19 therapies: associations with length of hospital stay, mortality and related costs Effect of hydroxychloroquine, azithromycin and lopinavir/ritonavir on the QT corrected interval in patients with COVID-19 Hydroxychloroquine versus lopinavir/ritonavir in severe COVID-19 patients: results from a real-life patient cohort Lack of viral clearance by the combination of hydroxychloroquine and azithromycin or lopinavir and ritonavir in SARS-CoV-2-related acute respiratory distress syndrome Retrospective multicenter cohort study showsearly interferon therapy is associated with favorable clinical responses in COVID-19 patients The dual role of anti-viral therapy in the treatment of coronavirus disease 2019 Comparison of therapeutic effects between monotherapy and combination therapy of lopinavir/ritonavir and Arbidol in COVID-19 Retrospective study on the effectiveness and safety of lopinavir/ritonavir in the treatment of coronavirus disease 2019 Clinical effect of lopinavir/ritonavir in the treatment of COVID-19 Clinical efficacy and safety of lopinavir/ritonavir combined with other antiviral in the treatment of coronavirus disease 2019 (COVID-19) Efficacies of lopinavir/ritonavir and arbidol in the treatment of novel coronavirus pneumonia Clinical application and curative effect analysis of lopinavir/ritonavir in severe/critical patients with COVID-19 Bias caused by sampling error in meta-analysis with small sample sizes Small studies may overestimate the effect sizes in critical care meta-analyses: a meta-epidemiological study Remdesivir, lopinavir, emetine, and homoharringtonine inhibit SARS-CoV-2 replication in vitro Small molecules targeting severe acute respiratory syndrome human coronavirus Comparative antiviral efficacy of viral protease inhibitors against the novel SARS-CoV-2 in Vitro Epub ahead of print) future science group Efficacy of lopinavir-ritonavir combination therapy for the treatment of hospitalized COVID-19 patients Meta-Analysis • An in vitro study which concluded that the serum concentration of lopinavir and ritonavir may not be sufficient for inhibiting the actions of SARS-CoV-2 Clinical pharmacology and biopharmaceutics reviews, center for drug evaluation and research Binding of lopinavir to human alpha1-acid glycoprotein and serum albumin • An observational study highlighting the high incidence of drug-drug interactions in COVID-19 patients treated with lopinavir-ritonavir therapy Lopinavir-ritonavir: a new protease inhibitor Drug-induced liver injury associated with antiretroviral therapy that includes HIV-1 protease inhibitors Pharmacokinetics and hepatotoxicity of lopinavir/ritonavir in non-cirrhotic HIV and hepatitis C virus (HCV) co-infected patients High rate of major drug-drug interactions of lopinavir-ritonavir for COVID-19 treatment Antiretroviral agents Structure and mechanism of the complex between cytochrome P4503A4 and ritonavir Lopinavir/ritonavir in the treatment of HIV-1 infection: a review Meyler's Side Effects of Drugs: The International Encyclopedia of Adverse Drug Reactions and Interactions COVID-19) treatment guidelines Guidelines on the treatment and management of patients with COVID-19 In vitro metabolism of dexamethasone (DEX) in human liver and kidney: the involvement of CYP3A4 and CYP17 (17,20 LYASE) and molecular modelling studies Cushing syndrome due to ritonavir-fluticasone interaction Lopinavir and ritonavir tablets, oral use, documentation, president's emergency plan for AIDS relief (PEPFAR) database Interaction between ritonavir and statins Atrial fibrillation and human immunodeficiency virus type-1 infection: a systematic review. Implications for anticoagulant and antiarrhythmic therapy COVID-19 treatment guidelines -antiviral therapy IDSA guidelines on the treatment and management of patients with COVID-19 COVID-19 clinical trials: quality matters more than quantity The carnage of substandard research during the COVID-19 pandemic: a call for quality