key: cord-0991981-dkhl9pjp
authors: Bulterys, Philip L.; Garamani, Natasha; Stevens, Bryan; Sahoo, Malaya K.; Huang, ChunHong; Hogan, Catherine A.; Zehnder, James; Pinsky, Benjamin A.
title: Comparison of a Laboratory-Developed Test Targeting the Envelope gene with three Nucleic Acid Amplification Tests for Detection of SARS-CoV-2
date: 2020-05-08
journal: J Clin Virol
DOI: 10.1016/j.jcv.2020.104427
sha: 6c40992590dba718babc9530b2142b8675e96428
doc_id: 991981
cord_uid: dkhl9pjp

ABSTRACT Background Numerous nucleic acid amplification tests, including real-time, reverse transcription PCR (rRT-PCR) and isothermal amplification methods, have been developed to detect SARS-CoV-2 RNA, including many that have received emergency use authorization (EUA). There is a need to assess their test performance relative to one another. Objectives The aim of this study was to compare the test performance of a high complexity laboratory-developed rRT-PCR EUA from Stanford Health Care (SHC) targeting the SARS-CoV-2 envelope (E) gene with other tests: the Atila isothermal amplification assay targeting the nucleocapsid (N) gene and open reading frame 1ab (ORF1ab), the Altona E and spike (S) multiplex, real-time RT-PCR, and the US Centers for Disease Control and Prevention (CDC) N1 and N2 rRT-PCRs. Study Design A diagnostic comparison study was performed by testing nasopharyngeal samples from persons under investigation for coronavirus disease 2019 (COVID-19). Assay performance was assessed by percent agreement and Cohen’s kappa coefficient. Results Positive percent agreement with the SHC EUA reference assay was 82.8% (95% confidence interval (CI) 65.0 to 92.9) for Atila, 86.7% (95% CI 69.7 to 95.3) for the Altona E and S targets, and 86.7% (95% CI 69.7 to 95.3) and 90.0% (95% CI 73.6 to 97.3), for the CDC N1 and N2 targets, respectively. All assays demonstrated 100% negative percent agreement. Kappa coefficients ranged from 0.86 to 0.92, indicating excellent agreement. Conclusions Performance was comparable among the SARS-CoV-2 nucleic acid amplification methods tested, with a limited number of discrepancies observed in specimens with low viral loads.

and S targets, and 86.7% (95% CI 69.7 to 95.3) and 90.0% (95% CI 73.6 to 97.3), for the CDC N1 and N2 targets, respectively. All assays demonstrated 100% negative percent agreement.

Kappa coefficients ranged from 0.86 to 0.92, indicating excellent agreement.

Performance was comparable among the SARS-CoV-2 nucleic acid amplification methods tested, with a limited number of discrepancies observed in specimens with low viral loads.

Accurate diagnostics for the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) are critical to identifying and managing individuals with Coronavirus Disease-2019 (COVID-19).

The current standard of care for diagnosis of active infection is the detection of viral RNA from respiratory specimens by real-time, reverse transcription polymerase chain reaction (rRT-PCR). [1] [2] [3] [4] There are, however, a limited number of isothermal amplification methods that have been developed for the detection of SARS-CoV-2 RNA. One such isothermal assay is the Atila iAMP COVID-19 detection kit, which targets the open reading frame 1ab (ORF1ab) region and nucleocapsid (N) gene in a single channel and utilizes the detection RNase P in a separate channel as the internal control. To evaluate this isothermal method, as well as two additional rRT-PCR assays, the N1 and N2 components of the US Centers for Disease Control and Prevention (CDC) assay and a commercial multiplex assay targeting the envelope (E) and spike (S) genes (Altona Diagnostics), we tested nasopharyngeal samples from individuals under J o u r n a l P r e -p r o o f investigation for COVID-19 using the Stanford Health Care (SHC) Clinical Virology Laboratory EUA assay as reference. Given the need to rapidly deploy SARS-CoV-2 diagnostic tests in response to the COVID-19 pandemic, these data are important to inform laboratory decisionmaking and to guide clinical management.

The aim of this study was to assess the test performance of the Atila, Altona, and CDC assays compared to the SHC EUA for the qualitative detection of SARS-CoV-2.

This study was approved by the Stanford University Institutional Review Board (protocol #48973). 

Because reference testing was performed on freshly extracted nucleic acids and comparator testing was performed after at least one freeze-thaw of archived eluates, all discrepant samples J o u r n a l P r e -p r o o f (reference detected, comparator not detected) were re-extracted and those eluates tested without freeze-thaw. Total nucleic acids were extracted from 400 µL on the BioRobot EZ1 (Qiagen, Germantown, MD) using the EZ1 virus mini kit 2.0 according to the manufacturer's recommendations, and eluted in 60 µL buffer AVE.

Positive percent agreement (PPA), negative percent agreement (NPA) and associated 95% confidence intervals (CI) were performed with the SHC EUA serving as the reference method.

Cohen's kappa coefficient of qualitative results (detected/non-detected) between the two assays with 95% CI was also calculated. Cohen's kappa values greater than 0.81 were interpreted to indicate excellent agreement. 7

To compare the performance of SARS-CoV-2 nucleic acid amplification tests, extracts from 80 nasopharyngeal swab specimens (30 detected, 50 not detected) originally tested using the SHC These results highlight subtle differences in sensitivity, the relative importance of which may vary based on the patient population tested and how frequently low viral load specimens are expected. Slightly reduced sensitivity was observed in the Atila iAMP assay; five reference positive samples were not detected in the initial experiments and three were still not detected after re-extraction. Isothermal reverse transcription amplification methods are typically at least as analytically sensitive as rRT-PCR, so the explanation for this difference is not readily apparent.

Furthermore, Atila requires the highest nucleic acid eluate input volume (18 µL) of these tests and utilizes common SARS-CoV-2 gene targets, though the details of the oligos used in the assay are not disclosed. However, this isothermal method yields results approximately 1 hour more rapidly than the rRT-PCR thermal cycling methods evaluated in this study. The decision to choose one assay over another in this setting should assess the balance between advantages in turnaround time and instrument or reagent availability, and the drawbacks of potentially reduced sensitivity. These include consideration of the underlying patient population being tested, as well as the possibly significant clinical and public health implications of missed COVID-19 cases.

J o u r n a l P r e -p r o o f 

Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR

Triplex Real-Time RT-PCR for Severe Acute Respiratory Syndrome Coronavirus 2

Comparative Performance of SARS-CoV-2

Detection Assays using Seven Different Primer/Probe Sets and One Assay Kit

Comparison of the Panther Fusion and a Laboratory-developed Test Targeting the Envelope gene for Detection of SARS-CoV-2

Food and Drug Administration. Stanford Health Care Clinical Virology Laboratory SARS-CoV-2 test EUA Summary

The measurement of observer agreement for categorical data

We would like to thank the members of the Stanford Health Care Clinical Virology Laboratory for their continued hard work and dedication to patient care. They have responded to the extreme pressures of the COVID-19 pandemic with composure, grit, and verve.J o u r n a l P r e -p r o o f