key: cord-0912849-8t04g7hq
authors: Carroll, Anne; McNamara, Eleanor
title: Comparison and correlation of commercial SARS-CoV-2 real-time-PCR assays, Ireland, June 2020
date: 2021-02-11
journal: Euro Surveill
DOI: 10.2807/1560-7917.es.2021.26.6.2002079
sha: 9b41f98b35dfe3c39210b5d74ddf1c80459ed417
doc_id: 912849
cord_uid: 8t04g7hq

We report the performance of a variety of commercially available SARS-CoV-2 PCR kits, used in several different sites across Ireland to determine if C(t) values across platforms are comparable. We also investigate whether a C(t) value, a surrogate for calculated viral loads in the absence of viral culture of > 34 can be used to exclude SARS-CoV-2 infection and its complications. We found a variation in C(t )values from different assays for the same calculated viral load; this should be taken into consideration for result interpretation.

We report the performance of a variety of commercially available SARS-CoV-2 PCR kits, used in several different sites across Ireland to determine if C t values across platforms are comparable. We also investigate whether a C t value, a surrogate for calculated viral loads in the absence of viral culture of > 34 can be used to exclude SARS-CoV-2 infection and its complications. We found a variation in C t values from different assays for the same calculated viral load; this should be taken into consideration for result interpretation.

The interpretation of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) real-time (RT)-PCR tests presents multiple conundrums with respect to viral load to cause infection and be infectious, age, clinical phase (presymptomatic, symptomatic, asymptomatic, resolution, re-infection or persistent positivity), testing purposes (diagnostic or surveillance), trending of previous test results if available and use of test result (e.g. for infection prevention control or occupational health purposes). In recent months, various publications have suggested that the use of cycle threshold (C t ) values as surrogate for calculated viral load, may help in the management of patients [1] [2] [3] [4] .

In this study, we investigate if C t values obtained by a variety of commercially available SARS-CoV-2 PCR kits, used in several different sites across Ireland, are comparable across platforms. We also explore whether a C t value of > 34 [3] , in the absence of viral culture, can be used to exclude SARS-CoV-2 infection.

No patient data or specimens were used in this study, therefore, ethical approval was not required.

In April 2020, Quality Control for Molecular Diagnostics (QCMD) produced a SARS-CoV-2 external quality assessment (EQA) panel [5] . The panel contained eight samples of which five were positive for SARS-CoV-2. Laboratories participating in the EQA were given the panel of samples without the respective information on positivity or negativity. Each laboratory processed the samples as if the provided material was viral transport media (VTM) from a SARS-CoV-2 inoculated swab, and tested the EQA panel according to their RT-PCR of choice and own laboratory procedures, then returned a 'Detected' or 'Not detected' result to QCMD. Following submission of results from all participants, in June 2020, QCMD provided a report to all participants, now detailing the digital (d)PCR log 10 copies/mL of SARS-CoV-2 in the samples for reference purpose. The dPCR log 10 copies/mL for the five SARS-CoV-2 positive samples were 4.3, 3.3, 4.3, 5.3, and 2.3 respectively.

In this study, we analyse the results of 16 participating clinical diagnostic laboratories across Ireland in more detail, using in particular the C t values that they obtained with their RT-PCR assays for each of the five positive samples. For each laboratory, the C t values for the five samples and the corresponding dPCR log 10 copies/mL were employed to produce standard curves for each assay (or for each assay target). When several laboratories used a common assay, this allowed to assess the performance of the same assay across the platforms. Moreover, when laboratories used different assays, it was possible to compare outputs across assays. Six for 16 laboratories submitted data on more than one assay. In total nine assays with in total 15 gene targets were analysed (Table 1) .

There were a number of different RNA extraction systems, either prior to the RT-PCR or incorporated within this procedure (no external extraction for GeneXpert (Cepheid)). Each of these systems used a different amount of sample (200 μL to 750 μL), with a varying portion of the total recovered RNA as subsequent PCR template. A standard curve was created in Microsoft Excel for each assay (one for each gene target). The standard Table 1 Real-time PCR assays considered in the study, Ireland, June 2020 (n = 9 assays) 

Average and standard deviation of Ct values at calculated SARS-CoV-2 viral loads of 3log 10 to 8log 10 copies/mL, Ireland, June 2020 (n = 8 assays) a

Average C t @ 3log 10 (SD)

Average C t @ 4log 10 (SD)

Average C t @ 5log 10 (SD)

Average C t @ 6log 10 (SD)

Average C t @ 7log 10 (SD)

Average C t @ 8log 10 (SD) SARS-CoV-2: severe acute respiratory syndrome coronavirus 2; SD: standard deviation. a While a total of nine SARS-CoV-2 real-time PCR assays were used in the study-participating laboratories, the Abbott RealTime SARS-CoV-2 assay reports cycle number values, which are not equivalent to C t values and thus are not directly comparable. This assay was therefore excluded from the analysis. b Absolute values for Altona and Seegene targets as only data from one laboratory. curve was created by plotting the C t value for each of the five samples against the dPCR log 10 copies/mL provided by QCMD. R 2 values of all the assays ranged from 0.9497 to 0.9997, with a mean of 0.9885. From each standard curve, estimated viral load (log 10 copies per mL) was extrapolated (using the equation of the individual standard curve) for each C t value. The Abbott RealTime SARS-CoV-2 assay reports cycle number (CN) values, which are not equivalent to C t values and thus are not directly comparable [6] and was therefore excluded from further analysis.

Mean C t values for all assays and standard deviations were calculated across a range of log 10 copies per mL values (Table 2 and 3) . Where only one laboratory tested an assay, absolute values were used for comparison. Correlation of C t results between the same assays used across different sites was good for all assays (mean: 1.6; standard deviation: 0-5.1) (Figure, Table 2 ). Data from all assays correlated with the internationally recognised 3.3 cycle difference for every 1log 10 copies/ mL change in viral load. However, there was a wide variation in C t values for different assays for the same viral loads, 6.5 cycle difference (31-37.5) at 3log 10 down to 4 cycle difference (22-26) at 6log 10 . But the range difference in C t values between assays was stable across all log values (Figure) . These data demonstrate that reporting C t values per se can be misleading and is non comparative between different assays, unless the C t value is correlated with the calculated viral load for the particular assay used and also reported.

Tom et al. 2020 [3] noted that the issue of high C t values can be problematic for clinicians, especially when there are less than 100 copies of the virus present, as this could reflect presymptomatic, early infection, late infection, persistent positivity or nonviable virus. From our data even at a C t as high as 37, six of eight assays had at least one gene target correlating to calculated viral load of ≥ 100 copies/mL, although our study did not include viral culture, nor infer cultureability. The C t cut off of 34 described by La Scola et al. [7] at which they propose patients can be discharged from isolation may need to be considered cautiously, as our data show that a C t value of 34 has a range of calculated viral loads from 2.19log 10 to 3.89log 10 (equivalent to approximately 150 -> 7000 viral copies per mL (Table  3) .

Our National coronavirus disease (COVID-19) guidelines [1, 2] also describe the difficulty of interpreting positive 'high' (> 30) C t PCR results from asymptomatic individuals, they too suggest that a C t of 34 equates to < 100 copies/mL, however, our data indicate that the Table 3 Average SARS-CoV-2 viral loads of log 10 copies/mL and standard deviation at selected C t values, Ireland, June 2020 (n = 8 assays) a Ct Average log 10 (standard deviation) (copies/mL) severe acute respiratory syndrome coronavirus 2. a While a total of nine SARS-CoV-2 real-time PCR assays were used in the study-participating laboratories, the Abbott RealTime SARS-CoV-2 assay reports cycle number values, which are not equivalent to C t values and thus are not directly comparable. This assay was therefore excluded from the analysis. b Absolute values for Altona and Seegene targets as only data from one laboratory.

C t value could be up to 38 for a calculated viral load of 100 copies/mL (data not shown).

A number of different RNA extraction systems were used by the participating laboratories, the effect this has on the results is unquantifiable. While it is a limitation of this study, our results support the view of Chik-Yan et al. [8] who state that differences in C t values may be due to differences in specimen source or preparation or differences in cycling parameters and reagents, even though there is no significant difference in sensitivity. Indeed, for our data there was good correlation in C t results using the same assay at different sites. Another limitation of the current study is that it investigated a small number of laboratories, with only one to six laboratories using the same assay (giving either absolute C t values or averages based on small number of replicates). A larger study in the future would be useful to support these results.

Consensus on the correlation between C t value and disease severity has not been reached, Sang Hyun Ra et al. [9] found that there was no significant difference in mean C t values from symptomatic or asymptomatic cases, whereas Salvatore et al. [4] reported higher C t values in asymptomatic individuals. Likewise Prubelli et al. [10] identified an increase in C t values that correlated with a decrease in severe cases in Italy. While diagnostic test results play a role in identification and aid management of infected individuals, it is imperative to have a thorough understanding of the performance characteristics of individual PCR assays to aid the accurate interpretation of results [11] .

Using C t values to influence patient management is complex and must be done with caution. Including the C t value on positive results may be confusing and a While a total of nine SARS-CoV-2 real-time PCR assays were used in the study-participating laboratories, the Abbott RealTime SARS-CoV-2 assay reports cycle number values, which are not equivalent to C t values and thus are not directly comparable. This assay was therefore excluded from the analysis.

b When more than one laboratory used an assay (or targets in the assay), the average C t s obtained across the laboratories are presented for each assay (or assay target).

Average C t values at calculated viral loads of 3log 10 to 8log 10 copies/mL. misleading [12] . With no clearly defined infectious dose for SARS-CoV-2, viral culture not available routinely and C t values differing by up to 6.5 cycles between platforms, one would question the value of the routine use of reporting C t values for patient management. This is particularly challenging in a community testing setting, where frequently there is no accompanying patient clinical information to aid interpretation. However, this may be ameliorated in an acute hospital testing environment with access to patient clinical data. In this setting, analysis of C t trends (either rising or decreasing) from repeat testing using the same assay may give more insight to an individual's disease progression or resolution.

To aid the clinical interpretative value of COVID-19 PCR results, we agree with Tom et al. that binary reporting ('Detected' or 'Not detected') could be enhanced by the additional reporting of C t value ranges in 'high', 'medium', 'low' categories . However, our data infer that such ranges if used, should be based on calculated viral loads of each assay used, not on absolute C t values. The viral loads for 'high', 'medium' and 'low' categories would need to be defined. This reporting would be more accurate and informative to aid clinical and public health decisions, particularly when considered in the context of individual clinical data.

Proposal for the management of weak positive (high Ct value) PCR results in the setting of mass testing of asymptomatic individuals for SARS-CoV-2

Guidance on the management of weak positive (high Ct value) PCR results in the setting of testing individuals for SARS-CoV-2 V1.2 22.12.2020. Ireland: HSE HPSC

To Interpret the SARS-CoV-2 Test, Consider the Cycle Threshold Value

Epidemiological Correlates of Polymerase Chain Reaction Cycle Threshold Values in the Detection of Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2)

Quality Control for Molecular Diagnostics (QCMD)

The implementation of a rapid sample preparation method for the detection of SARS-CoV-2 in a diagnostic laboratory in South Africa

Viral RNA load as determined by cell culture as a management tool for discharge of SARS-CoV-2 patients from infectious disease wards

Evaluation of the commercially available LightMix® Modular E-gene kit using clinical and proficiency testing specimens for SARS-CoV-2 detection

Upper respiratory viral load in asymptomatic individuals and mildly symptomatic patients with SARS-CoV-2 infection

Overall decrease in SARS-CoV-2 viral load and reduction in clinical burden: the experience of a hospital in northern Italy

Distribution of SARS-CoV-2 PCR Cycle Threshold Values Provide Practical Insight Into Overall and Target-Specific Sensitivity Among Symptomatic Patients

Challenges and Controversies to Testing for COVID-19

We would like to thank all contributing laboratories; The Coombe Women & Infants University Hospital, University Hospital Waterford, Our Lady's Hospital Navan, PHL-HSE-Dublin, Cork University Hospital, University Hospital Limerick, Midland Regional Hospital Portlaoise, University Hospital Kerry, Children's Health Ireland (CHI) at Crumlin, UCD National Virus Reference Laboratory, Letterkenny University Hospital, University Hospital Galway (UHG), The Meath Foundation Tallaght University Hospital, Tallaght, Midland Regional Hospital at Tullamore, The Mater Misericordiae University Hospital and Rotunda Hospital.

None declared.

Anne Carroll: study design, data collection, data analysis and manuscript preparation. Eleanor McNamara: study design, critically revised manuscript.This is an open-access article distributed under the terms of the Creative Commons Attribution (CC BY 4.0) Licence. You may share and adapt the material, but must give appropriate credit to the source, provide a link to the licence and indicate if changes were made.Any supplementary material referenced in the article can be found in the online version.