key: cord-0904128-trf131vo
authors: Mueller, Thomas; Kompatscher, Julia; La Guardia, Mario
title: Diagnostic performance of the Elecsys SARS‐CoV‐2 antigen assay in the clinical routine of a tertiary care hospital: Preliminary results from a single‐center evaluation
date: 2021-07-12
journal: J Clin Lab Anal
DOI: 10.1002/jcla.23906
sha: a9de6979b97ffa7cb0286c660271162a4f6baa77
doc_id: 904128
cord_uid: trf131vo

BACKGROUND: This report describes a manufacturer‐independent evaluation of the diagnostic accuracy of the Elecsys SARS‐CoV‐2 antigen assay from Roche Diagnostics in a tertiary care setting. METHODS: In this single‐center study, we used nasopharyngeal swabs from 403 cases from the emergency department and intensive care unit of our hospital. The reference standard for detecting SARS‐CoV‐2 was the reverse‐transcription polymerase chain reaction (RT‐PCR) assay. Cycle threshold (Ct) values were recorded for positive RT‐PCR assays. The index test was the Elecsys SARS‐CoV‐2 antigen assay. This electrochemiluminescence immunoassay produces results as cutoff index (COI) values, with values ≥1.00 being reported as positive. RESULTS: Of the 403 cases, 47 showed positive results in RT‐PCR assays. Of the 47 RT‐PCR‐positive cases, 12 showed positive results in the antigen assay. Of the 356 RT‐PCR‐negative cases, all showed negative results in the antigen assay. Thus, the antigen assay showed a sensitivity of 26% (95% CI, 14%‐40%) and specificity of 100% (95% CI, 99%‐100%). Analysis of the relationship between Ct values and COI values in the 47 RT‐PCR‐positive cases showed a correlation coefficient of −0.704 (95% CI, −0.824 to −0.522). The true‐positive rate of the antigen assay for Ct values of 15–24.9, 25–29.9, 30–34.9, and 35–39.9 was 100%, 44%, 8%, and 6%, respectively. CONCLUSIONS: The Elecsys SARS‐CoV‐2 antigen assay has a low sensitivity for detecting SARS‐CoV‐2 from nasopharyngeal swabs. Hence, we decided to not use this assay in the clinical routine of our hospital.

The RNA virus severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) causes coronavirus disease 2019 . 1 Infection with SARS-CoV-2 can be asymptomatic or may result in symptomatic disease ranging in severity from mild upper respiratory tract symptoms to severe pneumonia with respiratory failure and multiple organ failure. 1 The gold standard laboratory tests to detect SARS-CoV-2 from clinical specimens (eg, nasopharyngeal swabs, oropharyngeal swabs, and bronchoalveolar lavage fluid) are nucleic acid amplification tests (NAATs), mainly reverse-transcription polymerase chain reaction (RT-PCR) assays. 1, 2 Currently, a variety of NAATs are commercially available for use in routine clinical practice. 1, 3 However, since the testing capacity afforded by NAATs is insufficient to cope with the COVID-19 pandemic, various manufacturers have also developed rapid antigen immunoassays, which do not require skilled personnel and dedicated instrumentation, for detection of the virus from nasopharyngeal and oropharyngeal swabs. SARS-CoV-2 rapid point-of-care antigen tests have also been commercially available for some time. 1, 4 At present, antigen point-of-care tests in many countries help to ensure the necessary quantity of SARS-CoV-2 tests for their respective testing strategies, 1,4 but these tests have been criticized because of their lower clinical sensitivity in comparison with NAATs. 1, 4 Recently, Roche Diagnostics (Rotkreuz, Switzerland) launched a high-throughput antigen test for medical laboratories called "Elecsys SARS-CoV-2 antigen assay," which runs on the company's analyzers. We evaluated the diagnostic performance of the Elecsys SARS-CoV-2 antigen assay prior to its planned use in our clinical routine.

Herein, we report the results of our evaluation.

This report describes the findings of a single-center evaluation of the diagnostic accuracy of the Elecsys SARS-CoV-2 antigen assay as an index test in comparison with RT-PCR as the reference standard. Our manufacturer-independent evaluation was conducted from Pathology received 403 requests for simultaneous RT-PCR and antigen assays from the emergency department of the hospital and from the intensive care unit, which care for COVID-19 patients. These 403 requests pertained to 336 patients. In all 403 cases, two nasopharyngeal swabs were obtained simultaneously by skilled personnel, of which one was sent for the RT-PCR assay and the other was sent to run the antigen assay. We used the data from these 403 cases to evaluate the diagnostic performance of the Elecsys SARS-CoV-2 antigen assay. A referral to the ethics committee was not deemed necessary because the project was an assay validation/verification that was in line with good laboratory practice. Such evaluations are routinely performed in medical laboratories before introducing a new assay into the clinical routine.

The personnel from the emergency department of the hospital and from the intensive care unit used standard swabs and transport media from two different manufacturers, namely FLOQSwabs® (Ref. based on the detection of two gene targets, namely the amplification of the SARS-CoV-2 E and N2 genes. 1, 3 The limit of detection of this test was 250 copies/ml, and the time to result was 45 min. 1, 3 The Xpert Xpress SARS-CoV-2 test includes an early assay termination function, which can provide an earlier time to result for hightiter specimens if the signal from the target nucleic acid reaches a predetermined threshold before the full 45 PCR cycles have been completed.

Using the GeneXpert software (Cepheid, Sunnyvale, CA, USA), we considered positive RT-PCR results when the SARS-CoV-2 signal for the N2 nucleic acid target had a PCR cycle threshold (Ct) value of <40.0, irrespective of the signal for the E nucleic acid target. In contrast, when the Ct value for the SARS-CoV-2 N2 gene was ≥40.0, or when the results of RT-PCR testing were definitely negative (with reference to a positive result for the sample-processing control), we classified the result of the RT-PCR test as negative. Further, we categorized the results of RT-PCR tests that showed negative signals for the SARS-CoV-2 E and N2 genes as well as a negative signal for the sample-processing control as invalid; in these cases, we repeated the analysis.

Specimen collection and preparation for detection of the SARS- 09345302190) once daily at two COI levels. We allowed sample measurements only if the controls were within the defined limits.

We determined the limit of blank (LoB) as previously suggested 5 : Measurements were obtained with the SARS-CoV-2 extraction solution in replicates of 20 and calculated LoB = mean blank + 1.645 (SD blank ). Using this procedure, we found an LoB of 0.60 COI.

We evaluated the linearity of the Elecsys SARS-CoV-2 antigen assay according to the CLSI guideline EP6-A 6 using six different analyte concentrations. Fresh samples were used to prepare high-and low-concentration pools. We then conducted a direct dilution series with the low-and high-concentration patient sample pools in the following volume ratios (low-concentration pool +high-concentration pool): pool 1, low only; pool 2, 0.8 low +0.2 high; pool 3, 0.6 low aliquoted into ten plastic tubes for each concentration level and frozen at -80°C. We analyzed these samples in duplicate in two runs per day for 10 days within 2 weeks of sample collection. Within-run and total analytical precision (CV) were calculated using the CLSI double-run precision evaluation test. 7 The Elecsys SARS-CoV-2 antigen assay had a within-run CV of 3.3% and a total CV of 3.5% at a mean concentration of 1.12 COI (pool 1) and a within-run CV of 3.1% and a total CV of 5.7% at a mean concentration of 1.82 COI (pool 2).

We performed a purely descriptive statistical analysis by calculating the sensitivity, specificity, area under the ROC curve, positive likelihood ratio, negative likelihood ratio, positive predictive value, and negative predictive value for the Elecsys SARS-CoV-2 antigen assay against the reference standard. Sensitivity, specificity, positive and negative predictive values, and disease prevalence were expressed as percentages. The confidence intervals for sensitivity and specificity were the "exact" Clopper-Pearson confidence intervals. The confidence intervals for the likelihood ratios were calculated using the log method, as suggested by Altman et al. 8 Confidence intervals for the predictive values were the standard logit confidence intervals given by Mercaldo et al. 9 The area under the ROC curve was estimated using established procedures. [10] [11] [12] For correlation analysis, we calculated the Spearman correlation coefficient (rho) with a p-value and a 95% confidence interval (CI) for the correlation coefficient. Data analysis was performed using MedCalc software package MedCalc 17.2 (MedCalc Software Ltd, Ostend, Belgium). Figure 1 shows the respective scattergrams. In Table 2 

Although this is only a small single-center study, the main characteristics of the Elecsys SARS-CoV-2 antigen assay can be determined from our results. The Elecsys SARS-CoV-2 antigen assay had high specificity (it showed no false-positive results compared to the RT-PCR assay), but the assay showed lower sensitivity compared with the RT-PCR assay (it yielded many false-negative results). The assay showed a sensitivity of 26% in our cohort, which was fairly low. As 

Negative results In conclusion, it remains to be established whether the Elecsys SARS-CoV-2 antigen assay can be considered for detecting potentially infective individuals and thus for reducing the virus spread. If this is true, the Elecsys SARS-CoV-2 antigen assay could be useful for population screening of asymptomatic or pre-symptomatic individuals in accordance with the respective testing strategies of the authorities. In a tertiary care setting, however, the Elecsys SARS-CoV-2 antigen assay does not appear to be useful in its current form for clinical decision-making, in our opinion.

The authors would like to thank the staff of the Department of Emergency Medicine, the Department of Anesthesiology and

Intensive Care Medicine, and the Department of Clinical Pathology, Hospital of Bolzano, for their support in this project. We thank Maurizio Tait for the excellent technical assistance with the Elecsys SARS-CoV-2 antigen assay. We would also like to thank Fabio Rossi for extracting the necessary data from our laboratory information system (LIS).

None declared. 

The original anonymized data are available on request from the corresponding author.

https://orcid.org/0000-0002-5095-3964

SARS-CoV-2 infection and the COVID-19 pandemic emergency: the importance of diagnostic methods

IFCC interim guidelines on molecular testing of SARS-CoV-2 infection

Challenges in laboratory diagnosis of the novel coronavirus SARS-CoV-2. Viruses

Making sense of rapid antigen testing in severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) diagnostics. Diagnosis (Berl)

Limit of blank, limit of detection and limit of quantitation

Evaluation of the linearity of quantitative measurement procedures: a statistical approach; approved guideline. CLSI document EP6-A

Evaluation of precision performance of clinical chemistry devices; approved guideline. CLSI document EP5-A

Statistics with confidence: confidence intervals and statistical guidelines

Confidence intervals for predictive values with an emphasis to case-control studies

The meaning and use of the area under a receiver operating characteristic (ROC) curve

Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine

Receiver-operating characteristic curves and likelihood ratios: improvements over traditional methods for the evaluation and application of veterinary clinical pathology tests

SARS-CoV-2 infectivity by viral load, S gene variants and demographic factors and the utility of lateral flow devices to prevent transmission

Estimating infectiousness throughout SARS-CoV-2 infection course

Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection

Real-world clinical performance of commercial SARS-CoV-2 rapid antigen tests in suspected COVID-19: A systematic meta-analysis of available data as per

Clinical assessment of the Roche SARS-CoV-2 rapid antigen test. Diagnosis (Berl)

The comparative clinical performance of four SARS-CoV-2 rapid antigen tests and their correlation to infectivity in vitro

Diagnostic accuracy of two commercial SARS-CoV-2 antigen-detecting rapid tests at the point of care in community-based testing centers

Comparison of seven commercial SARS-CoV-2 rapid point-of-care antigen tests: a single-centre laboratory evaluation study

V-2-Public Health Laboratory team; Bavarian SARS-CoV-Public Health Laboratory team. Detection of the new SARS-CoV-2 variants of concern B.1.1.7 and B.1.351 in five SARS-CoV-2 rapid antigen tests (RATs)

Performance evaluation of a lateral flow assay for nasopharyngeal antigen detection for SARS-CoV-2 diagnosis

Development and clinical evaluation of an immunochromatography-based rapid antigen test (GenBody™ COVAG025) for COVID-19 diagnosis

Diagnostic performance of the Elecsys SARS-CoV-2 antigen assay in the clinical routine of a tertiary care hospital: Preliminary results from a single-center evaluation