key: cord-0724907-yzqf6f4w
authors: Hirotsu, Yosuke; Sugiura, Hiroki; Maejima, Makoto; Hayakawa, Miyoko; Mochizuki, Hitoshi; Tsutsui, Toshiharu; Kakizaki, Yumiko; Miyashita, Yoshihiro; Omata, Masao
title: Comparison of Roche and Lumipulse quantitative SARS-CoV-2 antigen test performance using automated systems for the diagnosis of COVID-19
date: 2021-06-01
journal: Int J Infect Dis
DOI: 10.1016/j.ijid.2021.05.067
sha: 86236f3cdfaf3ba7f4b7b9f2f6dd5b622e0f3c42
doc_id: 724907
cord_uid: yzqf6f4w

Background Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) continues to spread worldwide. Here, we evaluated the performance of two quantitative antigen (Ag) tests, the Roche and Lumipulse Ag test, using automated platforms. Methods We collected 637 nasopharyngeal swab samples from 274 individuals. Samples were subjected to quantitative reverse transcription PCR (RT-qPCR) as well as the Roche and Lumipulse Ag tests. Results When RT-qPCR was used as a reference, the overall concordance rate of the Roche Ag test was 77.1% (491/637) with 70.0% (341/487) sensitivity and 100% specificity (150/150);. When inconclusive results of the Lumipulse Ag test were excluded, the overall concordance rate of Lumipulse Ag test was 88.3% (467/529) with 84.8% (330/389) sensitivity and 97.9% (137/140) specificity. The overall concordance rate between the Roche and Lumipulse Ag tests was 97.9% (518/529) with 96.7% (322/333) sensitivity and 100% (196/196) specificity. Quantitative Ag levels determined using the Roche and Lumipulse Ag tests were highly correlated (R2 = 0.922). The Roche and Lumipulse Ag tests showed high concordance up to 9 days after symptom onset, with progressively lower concordance thereafter. Conclusions The Roche and Lumipulse Ag tests showed equivalent assay performance and represent promising approaches for diagnosis of coronavirus disease 2019.

Quantitative reverse transcription PCR (RT-qPCR) is a highly sensitive and specific assay and is considered the gold standard test for detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). RT-qPCR can detect viral RNA at very low copy number by amplification over 40 cycles. Therefore, RT-qPCR is a powerful tool for diagnosis of SARS-CoV-2 infection. Different approaches are, however, needed for infection control and public health measures [1] . RT-qPCR can detect viral RNA for several weeks in patients with coronavirus disease 2019 (COVID-19) [2] [3] [4] . Although shedding of viral RNA can be prolonged in clinical samples, the duration of viable virus shedding is relatively short [5] . Therefore, detection of viral RNA does not necessarily indicate the presence of live infectious viruses [5] .

Moreover, transmission events are thought to occur within a short period, likely a few days before and immediately after symptom onset [6] . Clinical samples generally do not contain culture-positive viruses (i.e., potentially "contagious" virus) from 8 to10 days after symptom onset [7] [8] [9] [10] [11] [12] .

We previously evaluated the accuracy of the Lumipulse antigen (Ag) test, an automated system that quantitatively measures nucleocapsid Ag using a chemiluminescent enzyme immunoassay [13, 14] . The proportion of samples with positive results using the J o u r n a l P r e -p r o o f Lumipulse Ag test decreased rapidly around 9 days after the onset of symptoms [13] , suggesting that this assay may be suitable for diagnosing COVID-19 patients when infectious viruses are present. Recently, Roche launched another Ag test platform for detecting nucleocapsid Ag. In this study, we compared the performance of the Roche and Lumipulse Ag tests to that of RT-qPCR using a panel of 637 samples. 

All nasopharyngeal swab samples were collected using cotton swabs and placed in 3 mL of viral transport media (VTM) obtained from Copan Diagnostics (Murrieta, CA, USA), and 700 μL of the VTM were used for the Lumipulse Ag test immediately after sample collection. The residual VTM was temporarily stored at 4°C and 200 μL of the VTM were used for nucleic acid extraction within 2 hours after sample collection. 300 μL of freeze-thaw VTM were used for Roche Ag test.

The Ag levels were determined quantitatively with the Lumipulse SARS-CoV-2 Ag test (Fujirebio, Inc., Tokyo, Japan) as we previously reported [13, 14] . In brief, 700 μL of the VTM samples were briefly vortexed, transferred into a sterile tube, and centrifuged at 2,000 ×g for 5 min. Aliquots (100 μL) of the supernatant were used for testing on the LUMIPULSE G600II automated system (Fujirebio). For samples with an Ag level > 5,000 pg/mL, the samples were diluted with the kit diluent and re-tested, and the Ag level was calculated were re-tested. Samples with an Ag level ≥ 10 pg/mL were considered positive, samples with ≥ 1.0 pg/mL and < 10.0 pg/mL were labeled inconclusive, while a result of ≤1.0 pg/mL was considered negative as per the manufacturer's guidelines.

To measure antigen levels, samples were subjected to the Elecsys ® SARS-CoV-2 Antigen Assay on a cobas ® 8000 (e801 module) automated platform (Roche, Basel, Switzerland) in accordance with the manufacturer's protocol with minor modifications. In brief, we transferred 300 µL of VTM into the sample cup (Hitachi, Tokyo, Japan) and added 30 µL of SARS-CoV-2 Extraction Solution C (Roche). After mixing for 5 seconds, the mixture was incubated for 5 minutes at room temperature. A 30 µL aliquot of the mixture was used for measuring antigen levels. This assay uses the electrochemiluminescence immunoassay principle. Samples with a cut off index (COI) of <1.0 were considered negative and those with a COI ≥1.0 were considered positive. Samples with COIs >17,000

were diluted with diluent and re-tested. COI values were then calculated taking the dilution factor into account. Of 487 PCR-positive samples, 4 samples (0.8%) were re-tested.

Total nucleic acid was isolated from the samples using the MagMAX Viral/Pathogen Nucleic Acid Isolation Kit (Thermo Fisher Scientific, Waltham, MA, USA) on the KingFisher Duo Prime System (Thermo Fisher Scientific) as we previously described [15, 16] . Briefly, we added 200 µL of VTM, 5 µL of proteinase K, 265 μL of binding solution, 10 μL of total nucleic acid-binding beads, 0.5 mL of wash buffer, and 0.5-1 mL of 80% ethanol to each well of a deep-well 96-well plate. The nucleic acids were eluted with 70 μL of elution buffer. The total nucleic acids were immediately subjected to RT-qPCR.

According to the protocol developed by the National Institute of Infectious Diseases (NIID) in Japan [17] , we performed one-step RT-qPCR to detect SARS-CoV-2. A threshold cycle (Ct) value was assigned to each PCR reaction and the amplification curve was visually assessed. According to the national protocol (version 2.9.1) [17] , we deemed a sample to be positive when a visible amplification plot was observed, whereas a sample was deemed negative when no amplification was observed.

The absolute copy number of viral loads was determined using serial diluted DNA control targeting the N gene of SARS-CoV-2 (Integrated DNA Technologies, Coralville, IA) as previously described [18] . The limit of detection of RT-qPCR using the primer/probe was considered as 2 copies according to the previous report [17] .

Sensitivity and specificity were calculated excluding inconclusive results of the Lumipulse Ag test. Fisher's exact test was used to assess differences among groups.

Values of p < 0.05 were considered statistically significant. Cohen's kappa (κ) coefficients and 95% confidence intervals (CI) were calculated using R version 3.6.2 (http://www.rproject.org/). Cohen's κ values greater than 0.81 were interpreted as almost perfect agreement [19] . Figure 1D ) and that of PCR-negative samples was −0.89 log10 pg/mL (range: −2.0 to 1.3 log10 pg/mL) ( Figure 1E ).

When RT-qPCR was used as a reference, the overall concordance rate of the Roche Ag test was 77.1% (491/637), sensitivity was 70.0% (341/487), and specificity was 100% (150/150) (κ = 0.524, 95% CI, 0.500-0.524) ( 

We next examined correlations between the COI values of the Roche Ag test, viral loads, and Ct values determined by RT-qPCR. Positive correlations were observed between the Roche COI value and viral load (R 2 = 0.805) and between the Roche COI value and Ct value (R 2 = 0.815) (Figure 2A) . Similarly, positive correlations were observed between Lumipulse Ag level and viral load (R 2 = 0.837) and between Lumipulse Ag level and Ct value (R 2 = 0.851) ( Figure 2B ), as we previously reported [13] .

Both the Roche and Lumipulse Ag tests target the nucleocapsid protein and quantitatively assess Ag levels. To examine whether quantitative Ag levels were correlated, we compared the results of the Roche and Lumipulse Ag tests. The Lumipulse Ag levels and Roche COI values were highly correlated (R 2 = 0.922) ( Figure 2C ). These results suggested that the Roche COI value reflected viral load and the amount of nucleocapsid Ag level with high accuracy.

To examine the relationship between viral load and the results of Ag tests, we calculated the number of viral copies used in each Ag test. The volume of VTM used for Ag test measurements was 100 µL for Lumipulse and 27.3 µL for Roche; therefore, the estimated number of viral copies per test is lower for the Roche Ag test. Table 2 ). The positive concordance rate gradually declined with decreasing viral load.

Compared with RT-qPCR, the Lumipulse Ag test was positive and showed 100% concordance for samples containing ≥5 log10 copies (131/131 samples), 99% concordance for samples containing 4-5 log10 copies (78/79), 84% for samples containing 3-4 log10 copies (61/73), 53% concordance for samples containing 2-3 log10 copies (37/70), 28% concordance for samples containing 1-2 log10 copies (21/75), and 3% concordance for samples containing <1.0 log10 copies (2/59) ( Table 2 ). These results indicated that RT-qPCR-positive samples with low viral loads were often judged as inconclusive using the Lumipulse Ag test (Table 2) .

This study included 468 RT-qPCR-positive samples collected from 123 symptomatic patients infected with SARS-CoV-2 and 19 RT-qPCR-positive samples from 16 asymptomatic patients. To examine the relationship between time since symptom onset and concordance rate, the 468 samples were assessed in more detail ( Table 3 ).

The concordance rates of RT-qPCR-positive samples were 93% vs. 89% (Roche vs. Lumipulse) in samples obtained 0-3 days after symptom onset, 82% vs. 83% in samples obtained 4-6 days after symptom onset, 75% vs. 72% in samples obtained 7-9 days after symptom onset, 47% vs. 45% in samples obtained 10-12 days after symptom onset, 39% vs. 36% in samples obtained 13-15 days after symptom onset, 14% vs. 14% in samples obtained 16-18 days after symptom onset, and 0% for both assays in samples obtained 19 days or more after symptom onset (Table 3) . Compared with RT-qPCR, there were no significant differences in the positive concordance rates of the Roche and Lumipulse Ag tests over this period (Table 3 , p > 0.05, Fisher's exact test).

In this study, we compared two Ag quantification tests (Roche and Lumipulse) Early detection and isolation of super-spreaders excreting live virus is important to reduce the spread of SARS-CoV-2. RT-qPCR is an extremely sensitive assay and can detect very tiny amounts of RNA in clinical specimens [21, 22] . Therefore, even in patients at the late stages of infection or after recovery, persistent viral excretion can been detected by RT-qPCR [21] . However, this may reflect the presence of non-infectious virus or viral debris. Meanwhile, Roche and Lumipulse Ag tests have high detection rates up to 9 days after symptom onset. On the basis of previous studies [4, 7, 12], live virus can be isolated for approximately 8-10 days after symptom onset. This coincides with the period that an infected individual can transmit the virus to others [3] . Therefore, the point when an Ag quantification test becomes negative may indicate a period of reduced infectivity [23] .

The Roche Ag test results in a negative or positive test result, whereas the Lumipulse Ag test results in a positive, inconclusive or negative result. The cut-off threshold for inconclusive result on the Lumipulse Ag test is between 1.0 pg/mL and 10 pg/mL. This threshold setting was shown to be useful in screening tests for the community, with high sensitivity and negative predicted value [24] . The inconclusive results were frequent, especially in samples with low viral load ( Figure 1A ). In our results, 90.7% (98/108) of the inconclusive results by Lumipulse Ag test were positive by RT-qPCR, but only 17.6%

(19/108) were positive by Roche Ag test (Table 1 ). In other words, most of the inconclusive results by Lumipulse Ag test were found to be negative by Roche Ag test. Although the relationship between sensitivity and specificity is a trade-off based on which threshold is set, the two tests are likely to differ in samples with low viral load. These results provide useful insight into the interpretation of the results of the two Ag quantification tests.

Ag tests have limitations. The sensitivity of Ag tests is lower than that of RT-qPCR. Therefore, it may be impossible to identify an infected patient at the very early or later phases of infection using Ag tests. However, it is sometimes difficult to detect the virus even by RT-qPCR. For example, multiple mutations have accumulated in circulating SARS-CoV-2 variants compared with the original strain identified in Wuhan, China [25] [26] [27] [28] [29] [30] [31] . These mutations inhibit PCR amplification, especially if the mutation occurs at the annealing location of primer and/or probe [32, 33] . Thus, RT-qPCR may not be able to detect all variants, and therefore it would be optimal to conduct both PCR and Ag tests to complement the shortcomings of each assay and enable accurate COVID-19 diagnosis.

In summary, quantitative Ag tests using automated systems offer rapid results, which can be used for prompt implementation of infection control and isolation measures.

Rapid and accurate decision making is of great importance in public health and will help to control the spread of SARS-CoV-2. 

YH contributed to study design, data collection, data analysis and writingreview & editing.

HS and MM contributed to sample preparation and data collection.

MH and HM contributed to supervision.

TT, YK and YM contributed to providing resources.

MO contributed to supervision and writingreview & editing.

☒ The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. 

Clarifying the evidence on SARS-CoV-2 antigen rapid tests in public health responses to COVID-19. The Lancet

Analysis of a persistent viral shedding patient infected with SARS-CoV-2 by RT-qPCR, FilmArray Respiratory Panel v2.1, and antigen detection

Shedding of infectious virus in hospitalized patients with coronavirus disease-2019 (COVID-19): duration and key determinants

SARS-CoV-2 Virus Culture and Subgenomic RNA for Respiratory Specimens from Patients with Mild Coronavirus Disease

Nasopharyngeal Swabs from 1,033 Patients using the LUMIPULSE SARS-CoV-2

Antigen Test: Comparison with RT-qPCR

Antigen Test for COVID-19 Infection with Quantitative RT-PCR using 313

Pooling RT-qPCR testing for SARS-CoV-2 in 1000 individuals of healthy and infection-suspected patients

Environmental cleaning is effective for the eradication of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in contaminated hospital rooms: A patient from the Diamond Princess cruise ship

Development of Genetic Diagnostic Methods for Novel Coronavirus 2019 (nCoV-2019) in Japan

Double-quencher probes improve detection 20

The dynamic change of antibody index against Covid-19 is a powerful diagnostic tool for the early phase of the infection and salvage PCR assay errors

Interpreting Diagnostic Tests for SARS-CoV-2

Detection of SARS-CoV-2 in Different Types of Clinical Specimens

Antigen-Based Testing but Not Real-Time Polymerase Chain Reaction Correlates With Severe Acute Respiratory Syndrome Coronavirus 2 Viral Culture

Evaluation of Lumipulse® G SARS-CoV-2 antigen assay automated test for detecting SARS-CoV-2 nucleocapsid protein (NP) in nasopharyngeal swabs for community and population screening

GISAID: Global initiative on sharing all influenza data -from vision to reality

Nextstrain: real-time tracking of pathogen evolution

European Centre for Disease Prevention and Control: Rapid increase of a SARS-CoV-2 variant with multiple spike protein mutations observed in the United Kingdom

Emergence and rapid spread of a new severe acute respiratory syndrome-related coronavirus 2 (SARS-CoV-2) lineage with multiple spike mutations in South Africa

Genomic characterisation of an emergent SARS-CoV-2 lineage in Manaus: preliminary findings

Discovery of a SARS-CoV-2 variant from the P.1 lineage harboring K417T/E484K/N501Y mutations in Kofu

Household transmission of SARS-CoV-2 R.1 lineage with spike E484K mutation in Japan

Two-step strategy for the identification of SARS-CoV-2 variant of concern 202012/01 and other variants with spike deletion H69-V70

Laboratory Action Plan for Emerging SARS-CoV-2 Variants

We thank the medical and ancillary hospital staff and the patients for consenting to participate in the study. We also thank Ryotaro Noguchi, Hiromichi Yamaji and Shogo Sato (Roche) for technical help and providing the antigen kits. We thank Edanz (https://enauthor-services.edanz.com/ac) for editing a draft of this manuscript.J o u r n a l P r e -p r o o f

The authors have no conflicts of interest to declare.