key: cord-0692209-rhhgkru8
authors: Glele, Ludwig Serge Aho; Simon, Emmanuel; Bouit, Camille; Serrand, Maeva; Filipuzzi, Laurence; Astruc, Karine; Kadhel, Philippe; Sagot, Paul
title: Association between SARS-Cov-2 infection during pregnancy and adverse pregnancy outcomes: a re-analysis of the data reported by Wei et al. (2021)
date: 2022-02-16
journal: Infect Dis Now
DOI: 10.1016/j.idnow.2022.02.009
sha: 8328cc54ea215709d4e0eb94e25fdea84ce1c0b0
doc_id: 692209
cord_uid: rhhgkru8

Objectives and background: Wei et al. have published a meta-analysis (MA) which aimed to evaluate the association between SARS-CoV-2 infection during pregnancy and adverse pregnancy outcomes. Using classical random-effects model, they found that SARS-CoV-2 infection was associated with preeclampsia, preterm birth and stillbirth. Performing MA with low event rates or with few studies may be challenging insofar as MA relies on several within and between-study distributional assumptions. The objective was to assess the robustness of the results provided by Wei et al. Methods: We performed a sensitivity analysis using frequentist and Bayesian meta-analysis methods. We also estimated fragility indexes. Results: For eclampsia, the confidence intervals of most frequentist models contain 1. All beta-binomial models (Bayesian) lead to credible intervals containing 1. The prediction interval, based on DL method, ranges from 0.75 to 2.38. The fragility index is 2 for the DL method. For preterm, the confidence (credible) intervals exclude 1. The prediction interval is broad, ranging from 0.84 to 20.61. The fragility index ranges from 27 to 10. For stillbirth, the confidence intervals of most frequentist models contain 1. Six Bayesian MA models lead to credible intervals containing 1. The prediction interval ranges from 0.52 to 8.49. The fragility index is 3. Conclusion: Given the available data and the results of our broad sensitivity analysis, we can suggest that SARS-CoV-2 infection during pregnancy is associated with preterm, and that it may be associated with preeclampsia. For stillbirth, more data are needed as none of the Bayesian analyses are conclusive.

J o u r n a l P r e -p r o o f Results: For eclampsia, the confidence intervals of most frequentist models contain 1. All betabinomial models (Bayesian) lead to credible intervals containing 1. The prediction interval, based on DL method, ranges from 0.75 to 2.38. The fragility index is 2 for the DL method.

For preterm, the confidence (credible) intervals exclude 1. The prediction interval is broad, ranging from 0.84 to 20.61. The fragility index ranges from 27 to 10. For stillbirth, the confidence intervals of most frequentist models contain 1. Six Bayesian MA models lead to credible intervals containing 1. The prediction interval ranges from 0.52 to 8.49 . The fragility index is 3.

Given the available data and the results of our broad sensitivity analysis, we can suggest that SARS-CoV-2 infection during pregnancy is associated with preterm, and that it may be associated with preeclampsia. For stillbirth, more data are needed as none of the Bayesian analyses are conclusive.

Wei et al. [1] recently published a systematic review and meta-analysis (MA) which aimed to "evaluate the association between SARS-Cov-2 infection during pregnancy and adverse pregnancy outcomes".

The authors show that SARS-CoV-2 infection is associated with preeclampsia, preterm birth and stillbirth. While they stated that they used Mantel-Haenszel method, the results of Figure 2 are related to a random-effects inverse-variance model, with DerSimonian-Laird estimate of tau² and continuity correction.

Higgin's I2 was used to assess heterogeneity. Although this approach is widely mentioned, the point estimate I2 should be interpreted cautiously when a MA has few studies [2] , and the confidence interval should be given.

MA relies on several within and between-study distributional assumptions that are sometimes hidden [3] .

Several methods are available to assign weights in meta-analyses, e.g. Mantel-Haenszel for fixed-effect MA or inverse-variance for fixed-effect or random-effects MA. For random-effects MA, there are several ways to estimate between-study variance, e.g. DerSimonian and Laird (DL), restricted maximum likelihood (REML). Finally, there are different ways to estimate the confidence interval for the summary effect (e.g. Wald or Hartung-Knapp-Sidik-Jonkman (HKSJ) method). The same applies to the confidence interval for between-study variance (e.g. Q-Profile method). See [4, 5] for more information.

Performing MA with low event rates or with few studies, or both, may be challenging. For example, in this case estimating between-study heterogeneity is difficult, and inaccurate estimation of this heterogeneity may lead to overly narrow confidence intervals.

Statistical methods can never completely resolve the issue of sparse data. Different methods may yield different results, and using a suboptimal approach may lead to erroneous conclusions [6] .

In order to assess the robustness of the results reported by Wei et al. [1] , we performed a sensitivity analysis using a range of frequentist and Bayesian meta-analysis methods. We also estimated fragility indexes (and Fragility Quotient, which is fragility index divided by the total study sample size). As an intuitive measure of the robustness of a trial, it is a number indicating how many patients would be required to convert a trial from being statistically significant to non-significant [7] . This index has been adapted for MA [8] and is currently being refined methodologically [9] . We estimated FI with the DL method and restricted maximum likelihood with Knapp-Hartung (KNHA) test.

Wei et al. [1] reported three outcomes: preeclampsia, preterm and stillbirth. We focused on preeclampsia and stillbirth, for which data are sparse.

For frequentist MA, several methods were used [10, 11] , including that of Hemni and Copas, which was found to be less sensitive to publication bias than the DL method [12, 13] (see tables). Although not optimal, continuity correction (0.5) was applied to studies with zero cells, the objective being to compare our results with the work of Wei et al. [1] .

In accordance with IntHout et al [14] , we have also presented prediction interval, but only for the model used by Wei et al. [1] . It is the interval within the effect size of which a new study would be categorized, if this study were selected at random from the same population as the studies already included in the MA.

For Bayesian MA, we used a binomial-normal hierarchical model (BNHM), i.e., modelling probabilities of success in each group [15] , instead of modelling estimates of log odds-ratios directly (normal-normal model) for the between-trial heterogeneity. We also used a beta-binomial (BB) model, which has shown good statistical properties for meta-analysis of sparse data [16, 17] . With these approaches, no continuity correction (of any type) is required, insofar as, unlike the commonly used normal-normal hierarchical model, these models are based on exact distributional assumptions. Several priors were used as sensitivity analysis [18, 19] .

Statistical analyses were performed with Stata (frequentist framework) and R software (Bayesian framework).

All estimates are shown in Tables 1 to 4. In some cases, no estimation was possible, one example being permutations, when fewer than six studies were included in the meta-analysis. Under these conditions, Bayesian estimates are more appropriate.

For eclampsia in general (comparing patients with Covid-19 vs without Covid-19), the confidence intervals of most frequentist MA models contain 1 ( Table 1) .

The prediction interval, based on the DL method, ranges from 0.75 to 2.38.

The fragility index is 2 for the DL method used by the authors, as is is with the REML-HKSJ method.

As for the Bayesian approach, all three beta-binomial models of MA lead to credible intervals containing 1. For binomial normal models, credible intervals do not contain one, but the upper boundary of the interval is very close to 1.

For eclampsia, when comparing patients with severe vs mild Covid-19, frequentist sensitivity analysis confirms the authors' results, except for one model out of 12 (Kenward-Roger's model).

The prediction interval, based on the DL method, ranges from 0.84 to 20.61.

The fragility index is 2 for the DL method and 1 with the REML-HKSJ method.

When heterogeneity of 75% is taken into account, the difference between the two groups is no longer significant.

Concerning the Bayesian analysis, only the binomial-normal models confirmed the initial results reported by Wei et al. [1] . None of the three beta binomial models resulted in credible intervals excluding 1.

For preterm (global analysis comparing patients with Covid-19 vs without Covid-19, or subgroup   analysis comparing patients with symptomatic vs asymptomatic Covid-19) , all frequentist models used in our sensitivity analysis led to confidence or credible intervals excluding 1, except for the inverse variance heterogeneity model and the Hemni-Copas method (Table 3) .

However, the prediction interval based on the DL method is very broad, ranging from 0.84 to 20.61.

The fragility index ranges from 27 (global analysis with the DL method) to 10 (subgroup analysis comparing symptomatic vs asymptomatic patients, with the REML-HKNHA method). Details on these fragility indexes and quotients are given in Table 4 .

All Bayesian credible intervals also exclude 1.

Finally for stillbirth, the confidence intervals of most frequentist meta-analysis (MA) models contain 1 ( Table 2) .

The prediction interval based on the DL method ranges from 0.52 to 8.49.

The fragility index is 3 for the DL method and 3 with the REML-HKNHA method.

As for the Bayesian approach, all six MA models lead to credible intervals containing one.

We performed a large-scale sensitivity study taking into account different approaches (frequentist and Bayesian) and different statistical models, the objective being to estimate several parameters including odds ratio, I2, Tau and their respective confidence intervals.

With the exception of preterm infants, and regardless of the frequentist or Bayesian statistical approach applied, the confidence intervals of odds ratios frequently overlap 1 and consequently show no association between Covid-19 and eclampsia or stillbirth. The prediction intervals are very wide and for all three endpoints (eclampsia, stillbirth and preterm) contain 1. Finally, the fragility indexes for eclampsia and stillbirth are only 1, 2 or 3.

Concerning our Bayesian sensitivity analysis, our results are robust as regards variation of the priors.

Some authors have proposed the use of generalized linear mixed model (GLMM) to perform MA with sparse data [20] . We have not used GLMM. Here again, several approaches are available. We considered that our sensitivity analysis was sufficiently broad. Finally, we did not take into account the quality of the studies via risk-of-bias analysis, which is possible through meta-regression.

Given the available data and the results of our broad sensitivity analysis, we can suggest that SARS-CoV-2 infection during pregnancy is associated with preterm, and may be associated with preeclampsia. For stillbirth, more data are needed as none of the Bayesian analyses are conclusive.

This study is a meta-analysis.

Ethical Approval : not needed.

Ludwig Serge Aho Glele (LSAG) and Emmanuel Simon (EM) contributed to the conception and design of the study. 

The impact of COVID-19 on pregnancy outcomes: a systematic review and meta-analysis

The heterogeneity statistic I2 can be biased in small meta-analyses

When should meta-analysis avoid making hidden normality assumptions?

Methods to estimate the between-study variance and its uncertainty in meta-analysis

Cochrane Handbook for Systematic Reviews of Interventions

Practical guide to the meta-analysis of rare events

The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index

The statistical significance of meta-analyses is frequently fragile: definition of a fragility index for meta-analyses

Factors that impact fragility index and their visualizations

Likelihood-based random-effects meta-analysis with few studies: empirical and simulation studies

Methods to calculate uncertainty in the estimated overall effect size from a random-effects meta-analysis

A confidence interval robust to publication bias for randomeffects meta-analysis of few studies

Confidence intervals for random effects meta-analysis and robustness to publication bias

Plea for routinely presenting prediction intervals in meta-analysis

The binomial distribution of meta-analysis was preferred to model within-study variability

A comparison of methods for meta-analysis of a small number of studies with binary outcomes

Beta-binomial models for meta-analysis with binary outcomes: Variations, extensions, and additional insights from econometrics

Random-effects meta-analysis of few studies involving rare events

On weakly informative prior distributions for the heterogeneity parameter in Bayesian random-effects meta-analysis

Laplace approximation, penalized quasi-likelihood, and adaptive Gauss-Hermite quadrature for generalized linear mixed models: towards meta-analysis of binary outcome with sparse data

Perm: permutation; PL: Partial Likelihood (-B: with Bartlett's correction; -S: with Skovgaard's correction); REML: Restricted Maximum Likelihood; RO: DL with SJ robust variance estimator

BNHM: Binomial-Normal Hierarchical Model; VAG: Vague prior; BN(t): Binomial Normal (t distribution); WIP: Weakly Informative Prior