key: cord-0783752-kvqmjyi3
authors: Gili, Alessio; Paggi, Riccardo; Russo, Carla; Cenci, Elio; Pietrella, Donatella; Graziani, Alessandro; Stracci, Fabrizio; Mencacci, Antonella
title: Evaluation of automated test Lumipulse® G SARS-CoV-2 antigen assay for detection of SARS-CoV-2 nucleocapsid protein (NP) in nasopharyngeal swabs for community and population screening
date: 2021-02-26
journal: Int J Infect Dis
DOI: 10.1016/j.ijid.2021.02.098
sha: bf75a8fddfc59b5b2d4903db9f5263f8b433ee79
doc_id: 783752
cord_uid: kvqmjyi3

OBJECTIVES: To compare the Lumipulse® SARS-CoV-2 antigen test with the gold standard real-time reverse transcription-polymerase chain reaction (RT-PCR) for diagnosis of SARS-CoV-2 infection and to evaluate its role in screening programs. METHODS: Lumipulse® SARS-CoV-2 antigen assay was compared to the gold standard RT-PCR test, in a selected cohort of 226 subjects with suspected SARS-CoV-2 infection and its accuracy was evaluated. Subsequently, the test was administered to a real-life screening cohort of 1,738 cases. ROC analysis was performed to explore test features and cutoffs. All tests were performed in the regional reference laboratory in Umbria, Italy. RESULTS: In the selected cohort we observed 42.0% positive results at RT-PCR. The Lumipulse® system showed 92.6% sensitivity (95% CI 85.4-97.0%) and 90.8% specificity (95% CI 84.5-95.2%) at 1.24 pg/mL optimal cutoff. In the screening cohort, characterized by 5.2% prevalence of infection, Lumipulse® assay showed 100% sensitivity (95% CI 96.0-100.0%) and 94.8% specificity (95% CI 93.6-95.8%) at 1.645 pg/mL optimal cutoff. The AUC was 97.4%, NPV was 100% (95% CI 99.8-100.0%) and PPV 51.1% (95% CI 43.5-58.7%). CONCLUSIONS: Lumipulse® SARS-CoV-2 antigen assay can be safely employed in the screening strategies in small and large communities and in the general population.

On December 2019, several cases of unknown aetiology pneumonia in Wuhan City in Hubei Province were reported from China Health Authority (Lu et al., 2020) . Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was identified on January 7, 2020 (Hui et al., 2020) as the aetiologic agent of the disease. On January 30, World Health Organization declared SARS-CoV-2 outbreaks a Public Health Emergency of International Concern (Burki, 2020a ) and on March 11 declared Coronavirus Disease 2019 pandemia. To date, more than 107 million people and more than 2.4 million deaths were reported globally (WHO, 2020b) .

Currently, the gold standard for COVID-19 microbiological diagnosis is the detection of SARS-CoV-2 genetic targets in respiratory samples using molecular real-time reverse transcriptionpolymerase chain reaction (RT-PCR) test (WHO, 2020a) . The same test is used for the epidemiological surveillance of the infection, aiming to contain virus spreading in the community.

As individuals before symptom onset and individuals who never develop symptoms can be highly contagious (Huff and Singh, 2020) , extensive community testing is considered one of the cornerstones of control strategies of the spread of the infection. In spite of the high sensitivity and specificity, RT-PCR suffers from a series of limitations, such as the long time required to be performed, the need for dedicated equipment and highly specialized laboratory technicians, and high costs. Hence, there is a pressing need to introduce new diagnostic technologies, equally reliable, but at the same time rapid, easily suitable into the laboratory work-flow, and economically advantageous (ECDC, 2020a), for screening strategies based on extensive communities testing.

Recently, Lumipulse ® SARS-CoV-2 antigen assay (Fujirebio, Inc., Tokyo, Japan) automated test was introduced into the market. The system, capable of processing up to 120 samples per hour, is widely used in Japan for COVID-19 surveillance, and in August 2020 it obtained the European CE-IVD mark for in vitro diagnostic use. The system is based on the chemiluminescence enzyme immunoassay (CLEIA) technology, capable of detecting and quantitatively estimate the presence of the SARS-CoV-2 nucleocapsid protein (NP) in nasopharyngeal swabs or saliva. Hirotsu et al. have J o u r n a l P r e -p r o o f recently demonstrated that it can identify successfully SARS-CoV-2 infected patient with moderateto-high viral load (Hirotsu et al., 2020b) . The present study evaluates the possible role of the Lumipulse ® SARS-CoV-2 antigen assay in selected communities (e.g. health professionals, schools, residential and nursing home for the elderly) or population-wide screening for SARS-CoV-2 infection.

To this aim, the antigen assay was first evaluated on a small high prevalence selected series of 226 nasopharyngeal swabs (selected cohort), sent to the reference laboratory of the Umbria Italian Region (coverage about 870,000 people) at the start of the second epidemic wave and analyzed by RT-PCR. Subsequently, the antigen test was administered to a second unselected cohort (screening cohort), composed by 1,738 swabs resulting from real-life screening scenarios (e.g., schools, hospital healthcare workers, and other communities), and results were compared with those of RT-PCR.

J o u r n a l P r e -p r o o f

The Lumipulse ® assay was first tested on a selected series of 226 nasopharyngeal swabs (selected cohort), analyzed from 10 to 15 September 2020 at the Microbiology Unit of the Santa Maria della Misericordia Perugia General Hospital, that is the Umbria Regional Reference Laboratory for SARS-CoV-2 diagnosis. The selection of the swabs to be evaluated with the antigen assay was performed in order to include in the study a large number of samples, with 1 or more target genes detected, with high variability of Ct, as a proxy of viral load, to be able to evaluate the test in terms of sensitivity and specificity with adequate numbers for RT-PCR positive and negative cases.

Samples from both symptomatic and strict contacts individuals were included, while samples from patients already diagnosed with SARS-CoV-2 infection were excluded.

Subsequently, over the period 1 to 26 January 2021, the Lumipulse ® assay was employed in reallife screening strategies in a second unselected cohort (screening cohort) of 1,738 swabs, collected in schools, prisons, care homes for elderly, and from hospital healthcare workers surveillance programs. All swabs were also analyzed by RT-PCR test.

In the selected cohort, all samples were first tested for SARS-CoV-2 by RT-PCR, stored at 4° C, and then analyzed with the Lumipulse ® system within 24 hours. In the screening cohort, samples were analyzed with the antigen test, and soon after by RT-PCR. Swabs were collected in Universal Transport Medium (UTM, Copan, Brescia, Italy). For RT-PCR, samples were analyzed by the Allplex™ SARS-CoV-2 assay (Seegene, Seoul, South Corea). The test was performed according to the manufacturer's instructions, using 300 μL of UTM and 10 μL of the provided internal control (IC). The envelope (E) gene (specific of the subgenus Sarbecovirus), the nucleocapsid gene (N), J o u r n a l P r e -p r o o f and the RNA-dependent-RNA-polymerase (RdRP) gene (both specific of the SARS-CoV-2) were the target genes. The assay was considered valid if the Ct value of the IC was ≤ 40.

For the antigen test, after removing of the swabs, UTM tubes were centrifuged at 1,400 x g for 10 min, and loaded on the Lumipulse ® G1200 automated immunoassay analyzer (Fujirebio) to measure the NP antigen level with the Lumipulse ® SARS-CoV-2 antigen kit (Fujirebio) following the manufacturer's instruction. Briefly, the treatment solution and the sample were mixed and the mixture was dispensed into the anti-SARS-CoV-2 antigen monoclonal antibody-coated magnetic particle solution and then incubated for 10 min at 37 °C. After the first wash step, alkaline phosphatase-conjugated anti-SARS-CoV-2 antigen monoclonal antibody was added and incubated again for 10 min at 37 °C. After another wash step, the substrate solution was added, incubated for 5 min at 37 °C, and the amount of NP antigen (pg/mL) in the samples was determined in relation to the obtained luminescent signal.

The results obtained were analyzed using Stata Statistical Software (Release 16.1, College Station, Houston, TX: StataCorp LLC). Regarding the molecular test, analysis was performed using the cutoff of 35 Ct to discriminate samples positive for SARS-CoV-2 ( 35 for at least one of the target genes detected) from negative (>35 for all target genes detected), based on the accepted notion that subjects with SARS-CoV-2 positive samples with Ct >35 are not contagious (Binnicker, 2020; Bullard et al., 2020; Gupta, 2020; Singanayagam et al., 2020; Tom and Mina, 2020) . Moreover, the cutoff was inferred by the Italian Health Ministerial Circular no. 9774 of March 20, 2020, stating that "confirmation tests should be performed only for samples in which the result is difficult to interpret or the Ct in RT-PCR is greater than 35. In these cases it is recommended to repeat the test on a new sample" (Ministero della Salute, 2020).

Regarding the antigen assay, we considered the following cutoff values: 1.340 pg/mL, suggested by the manufacturer to discriminate samples positive for SARS-CoV-2 antigen NP from negative J o u r n a l P r e -p r o o f 8 samples, and 1.240 pg/mL and 1.645 pg/mL to optimize sensitivity and specificity of the test in the selected and screening cohort, respectively. Optimal cutoffs were obtained by Youden approach (area under ROC curve at cutoff 0.92 in selected cohort and 0.97 in screening cohort) and were confirmed also by Liu approach (Fluss et al., 2005; Liu, 2012; Youden, 1950) . The 1.645 pg/mL screening cohort cutoff was bootstrapped to estimate 95% confidence intervals (100 replications).

We calculated test sensitivity, specificity, ROC area, Positive Likelihood Ratio (LR+) and Negative Likelihood Ratio (LR-), Odds Ratio (LR+/LR-), Positive and Negative Predicted Value (PPV and NPV), prior and posterior probability (Odds) with 95% CI.

To calculate PPV and NPV in the 2.5% low-prevalence scenario, estimated by an Italian National seroprevalence study performed in June 2020 (ISTAT, 2020), we used values of sensitivity and specificity of the screening cohort, that was unselected and larger than selected cohort.

In the selected cohort, among 226 nasopharyngeal swabs, 116 (51.3%) were negative for all target genes, 95 (42.0%) were positive for 1, 2, or 3 target genes, with Ct 35. Other 15 (6.7%) samples were positive for 1 or more target genes with Ct >35 and were considered as negative. Median Ct for positive samples was 29. Table 1 Sensitivity and specificity were 90.5% and 91.6%, respectively, and LR+ reached 10.8. NPV and PPV were 93.0% and 88.7%, respectively. At the optimal cutoff of 1.240 pg/ml, sensitivity and NPV increased to 92.6% and 94.4%, respectively (Table 2) . Overall agreement raised to 91.6% (207/226 samples) and AUC to 91.7% (Figure 1) . Specificity, PPV, and LR+ decreased to 90.8%, 88.0%, and 10.1, respectively (Table 1 and Figure 2 ). Among 1,738 nasopharyngeal swabs, 1,644 (94.6%) were negative for all target genes, 90 (5.2%) were positive for 1, 2, or 3 target genes, with Ct 35. The remaining 4 (0.2%) samples were positive for 1 or more target genes with Ct >35 and were considered as negative. Median Ct for positive samples was 22. Table 1 compares RT-PCR and antigen assay results, according to the manufacturer cutoff of 1.340 pg/mL or the optimal cutoff of 1.645 pg/mL, obtained for this specific cohort, as described above (Figure 1 ). This optimal cutoff was not statistically different from that calculated for the selected cohort (95% CI 0.69-2.59 pg/mL).

At the 1.340 pg/ml cutoff value, overall agreement was 92.5% (1,608/1,738), AUC was 96.1%.

Sensitivity and specificity were 100% and 92.1% respectively; LR+ was 12.7, and NPV and PPV were 100% and 40.9%, respectively. At 1.645 pg/mL optimal cutoff overall agreement reached J o u r n a l P r e -p r o o f 95.1% (1,652/1,738), AUC was 97.4% (Figure 1 ). Sensitivity and NPV were 100%; specificity, PPV, and LR+ increased to 97.4%, 51.1%, and 19.2, respectively (Table 2 and Figure 2 ).

The performance of the test in 2.5% prevalence scenario, calculated on sensitivity and specificity (100% and 94.8%, respectively) of the screening cohort, that was unselected and larger than selected cohort, was estimated to be 100% NPV (95% CI 99.8-100%) and 32.9% PPV (95% CI 28.6-37.6%).

Assuming to test with RT-PCR all samples with antigen concentration >1.645 pg/mL to confirm infection, in a scenario with a prevalence of 2.5%, the estimated posterior positive probability is 33%, indicating that out of 100 samples tested for the NP antigen, 8 should have been evaluated by RT-PCR. Of the 8 samples, 2 would be positive for SARS-CoV-2 at the molecular test and 6 would result negative.

By the same assumption, in the screening cohort, 176 samples with antigen concentration >1.645 pg/mL would have been evaluated by RT-PCR and 86/176 (48.9%) would test negative for SARS-CoV-2 infection. To optimize the use of the Lumipulse ® antigen test and reduce the number of samples to be confirmed by RT-PCR in the routine laboratory practice, we searched for the cutoff value for optimal specificity and LR+ of the assay based on the screening cohort (i.e. a cutoff associated with a very high probability of a true positive result). The best LR+ (320.5) was found at the cutoff value of 10.4 pg/mL, with a specificity of 99.8%. According to this cutoff, in our screening cohort, the RT-PCR test could have been avoided for 74/176 samples with a reduction of 42% RT-PCR tests.

The main result of this study is that Lumipulse ® SARS-CoV-2 antigen assay, compared to RT-PCR from nasopharyngeal swabs, showed an excellent NPV for the presence of SARS-CoV-2 infection both in high and low prevalence scenarios. This result supports the use of this assay in selected high-risk communities and for community and population screening purposes.

In the low prevalence screening cohort, concordance between Lumipulse ® and RT-PCR was  94.8%, at the 1.645 optimal cutoff, and LR+ was 19.2, a figure which is excellent according to Deeks and Altman classification (Deeks and Altman 2004). We found a sensitivity 92.5% in both high-prevalence selected cohort and low prevalence screening cohort, remarkably higher than that found by Hirotsu et al. of 55.2% (Hirotsu et al., 2020b) . The difference could be explained by the fact that the study of Hirotsu et al. was performed in a population of hospitalized patients, mostly with low-viral load, already known to be infected with SARS-CoV-2. Moreover, many patients studied by Hirotsu et al. were probably in the late phase of the infection. Since our study was aimed to evaluate the performance of the test for screening purpose, we considered a population of nonhospitalized subjects, with no laboratory evidence of previous SARS-CoV-2 infection. We found a median Ct value of 29 in the selected cohort and 22 in the screening cohort, which suggests a viral load higher than that of Hirotsu's study (Hirotsu et al., 2020b) . Our results are in accordance with the optimal correlation between RT-PCR and Lumipulse ® antigen test found in the early phase of the infection, characterized by high viral load (Hirotsu et al., 2020a; Hirotsu et al., 2020b) . The performance of the antigen test was better in the screening cohort than in the selected one, which may reflect a high prevalence of early cases with high viral load diagnosed at screening during a phase of rapid increase of SARS-CoV-2 spread in our regional population. In addition, results from a semi-quantitative RT-PCR method, like the one used in our study, cannot be directly compared to those obtained with the quantitative method, used by Hirotsu et al., especially in case of low viral load, in the late phase of infection (Yu et al., 2020) .

As expected, specificity was lower than that found in the validation study of 99.6% (Hirotsu et al., 2020b) . We found a test specificity of 90.8% and 94.8%, according to our study best cutoffs, for the selected and screening study cohort, respectively (Figure 1 ). When the test was used in the low prevalence population, as expected, many cases with NP antigen >1.645 pg/mL had a falsely positive antigen test result (48.9% of all positive tests). However, two strategies can be adopted to reduce RT-PCR confirmation in case of limited availability of this resource: a. assume that the antigen test positivity is diagnostic of SARS-Cov-2 infection for high antigen concentrations, and b.

isolate all persons with a positive antigen test result (this choice may be costly in case of population screening).

We found that 74/176 (42%) RT-PCR tests to be performed to confirm a positive Lumipulse ® antigen test could have been omitted, based on the 10.4 ng/mL cutoff, which we could safely assume as diagnostic of SARS-CoV-2 infection, at a 99.8% specificity level. This cutoff is very close to that of 10.0 pg/mL proposed by the manufacturer to establish sample positivity for SARS-CoV-2 infection.

Both at the observed prevalence in our screening cohort (5.2%) and at the hypothetical 2.5% prevalence scenario, NPV remained >99.0%, an excellent value to be employed in a community or population screening. A screening based on this test could aim to greatly reduce or even eliminate transmission in the target communities as schools, retirement homes, clinics, prisons, and others in which it is difficult to guarantee strict observance of containment measures.

Gold standard RT-PCR test takes a minimum of 3-4 hours for results, needs specialized laboratory equipment and technicians, and has higher costs. On the other hand, the Lumipulse ® antigen assay is completely automated, can be easily introduced in laboratory routine workflow, and is capable of processing up to 120 samples per hour with a time-to-report of about 1 h. As accessibility, frequency, and sample to-answer time are crucial for effective surveillance of COVID-19 , our findings suggest that this antigen assay can be optimally employed to J o u r n a l P r e -p r o o f control the spreading of the virus in this pandemic under conditions in which molecular testing is not widely feasible.

Novel SARS-CoV-2 virus variants of potential concern with different mutations of the Spike protein have recently emerged (ECDC, 2021) . Detection ability of the NP antigen by the Lumipulse ® assay is not theoretically impacted by these variants. Indeed, preliminary results of genome sequencing of some positive samples included in our screening cohort showed that the antigen assay detected both P1 Brazilian and VOC 202012/01 UK variants, actively spreading in our Region in January 2021 (data not shown).

Diagnostic tests for SARS-CoV-2, including antigenic tests, are increasingly used for mass screening purpose in the different epidemic phases. In Wuhan city, at the end of the epidemic, a very large screening using nucleic acid testing on about 10 million people, found only 300 asymptomatic cases (Cao et al., 2020) . Other experiences have been made in Liverpool, UK (ECDC 2020b), Slovakia (Pavelka et al., 2020) , and Alto Adige, Italy (Euronews, 2020) . In Europe, lateral flow antigenic tests were recently introduced, during an active epidemic phase, with the purpose to reduce the effective reproduction number (Rt) and improve transmission control. However, mass screening based on such low sensitivity tests, although sustainable and affordable, may be ineffective because of false reassurance of infectious people testing negative (Iacobucci, 2020) .

Indeed, solid evidence of effectiveness for these screenings is still lacking and Slovakia recently was forced to adopt new lockdown measures shortly after the mass screening based on a lateral flow rapid test with a declared 30% false negative rate (Burki, 2020b) . To overcome the influence of low sensitivity on screening performance, it is necessary to repeat individual testing many times in close screening rounds Larremore et al., 2020) . Thus, the automated, fast and cheap Lumipulse ® assay could be a good alternative to lateral flow tests because of its higher sensitivity.

Studies reported saliva as a reliable sample for COVID-19 molecular diagnosis (Azzi et al., 2020; Yang et al., 2020; Yu et al.,2020) . It would be interesting to explore the performance of Lumipulse ® antigen test on this non-invasive specimen. 

Saliva is a reliable tool to detect SARS-CoV-2

Can the SARS-CoV-2 PCR Cycle Threshold Value and Time from Symptom Onset to Testing Predict Infectivity?

Predicting infectious SARS-CoV-2 from diagnostic samples

Coronavirus in China

Mass testing for COVID-19

Post-lockdown SARS-CoV-2 nucleic acid screening in nearly ten million residents of Wuhan, China

Diagnostic tests 4: likelihood ratios

Diagnostic testing and screening for SARS-CoV-2. ECDC

Population-wide testing of SARS-CoV-2: country experiences and potential approaches in the EU/EEA and the UK Stockholm

Assessment: Risk related to the spread of new SARS-CoV-2 variants of concern in the EU/EEA -first update

Covid-19, screening di massa in Alto Adige: testate oltre 320mila persone in 72 ore

Estimation of the Youden Index and its associated cutoff point

Strong Inverse Correlation Between SARS-CoV-2 Infectivity and Cycle Threshold Value

Analysis of a persistent viral shedding patient infected with SARS-CoV-2 by RT-qPCR, FilmArray Respiratory Panel v2.1, and antigen detection

Comparison of automated SARS-CoV-2 antigen test for COVID-19 infection with quantitative RT-PCR using 313 nasopharyngeal swabs, including from seven serially followed patients

Asymptomatic transmission during the COVID-19 pandemic and implications for public health strategies

The continuing 2019-nCoV epidemic threat of novel coronaviruses to global health -the latest 2019novel coronavirus outbreak in Wuhan, China

Covid 19: Mass population testing is rolled out in Liverpool

First Reports of SARS-CoV-2 Seroprevalence

Test sensitivity is secondary to frequency and turnaround time for COVID-19 surveillance

Classification accuracy and cut point selection

Outbreak of pneumonia of unknown etiology in Wuhan China: the mystery and the miracle

Rethinking Covid-19 Test Sensitivity -A Strategy for Containment

Update of contact tracing and laboratory diagnosis of SARS-CoV-2 infection

The effectiveness of populationwide, rapid antigen test based screening in reducing SARS-CoV-2 infection prevalence in Slovakia

Duration of infectiousness and correlation with RT-PCR cycle threshold values in cases of COVID-19

To Interpret the SARS-CoV-2 Test, Consider the Cycle Threshold Value

Laboratory testing for coronavirus disease (COVID-19) in suspected human cases: interim guidance

World Health Organization Coronavirus disease 2019 (COVID-19) Situation

Laboratory diagnosis and monitoring the viral shedding of 2019-nCoV infections

Index for rating diagnostic tests

Quantitative Detection and Viral Load Analysis of SARS-CoV-2 in Infected Patients

Table 2. Evaluation of Lumipulse ® antigen assay with 1.240 pg/mL and 1.645 pg/mL optimal cutoffs on selected cohort and screening cohort

AUC: Area Under the Curve; LR-: Negative Likelihood Ratio; LR+: Positive Likelihood Ratio

NPV: Negative Predictive Value; PPV: Positive Predictive Value; ROC: Receiver Operating Characteristic