key: cord-0737093-tab4yyhi
authors: Hirotsu, Yosuke; Maejima, Makoto; Shibusawa, Masahiro; Natori, Yume; Nagakubo, Yuki; Hosaka, Kazuhiro; Sueki, Hitomi; Amemiya, Kenji; Hayakawa, Miyoko; Mochizuki, Hitoshi; Tsutsui, Toshiharu; Kakizaki, Yumiko; Miyashita, Yoshihiro; Omata, Masao
title: Direct comparison of Xpert Xpress, FilmArray Respiratory Panel, Lumipulse antigen test, and RT-qPCR in 165 nasopharyngeal swabs
date: 2022-03-04
journal: BMC Infect Dis
DOI: 10.1186/s12879-022-07185-w
sha: 5d7fc1552ad88036a9d887c68f624c3ce3aed20c
doc_id: 737093
cord_uid: tab4yyhi

BACKGROUND: The nucleic acid amplification test (NAAT) and antigen test are approved diagnostic tests for COVID-19. In this study, we aimed to investigate the assay performance of two NAATs (Xpert Xpress SARS-CoV-2 and FilmArray Respiratory Panel) and a quantitative antigen test (Lumipulse). METHODS: One hundred and sixty-five nasopharyngeal swabs were subjected to Xpert, FilmArray, Lumipulse, and RT-qPCR assays. RESULTS: Of 165 samples, RT-qPCR showed 100 positives and 65 negatives. The Xpert had an overall agreement of 99.4% (95% confidence interval [CI]: 96.7–99.4%), sensitivity of 99% (95% CI: 96.8–99%), and specificity of 100% (95% CI: 96.6–100%). FilmArray had an overall agreement of 98.8% (95% CI: 95.9–98.8%), sensitivity of 98% (95% CI: 95.6–98%), and specificity of 100% (95% CI: 96.3–100%). Lumipulse had an overall agreement of 95.5% (95% CI: 91.8–95.5%), sensitivity of 92.3% (95% CI: 89.2–92.3%), and specificity of 100% (95% CI: 95.5–100%). The κ coefficient showed excellent agreement between each test and RT-qPCR. There was a high correlation between Xpert Ct values, RT-qPCR Ct values, viral loads and antigen level. CONCLUSIONS: Xpert Xpress and FilmArray Respiratory Panel exhibited an equivalent performance. The Lumipulse antigen test was slightly less sensitive than the NAATs, but showed high assay performance except for samples with low viral load. The Xpert Xpress, FilmArray Respiratory Panel and Lumipulse antigen tests offer rapid sample-to-answer data, allowing random access detection on automated devices.

The emergence of novel Variants of Concern (VOCs) is threatening human life [1] [2] [3] [4] [5] . In particular, the spread of the Alpha, Beta, Gamma, Delta and Omicron variant has led to an infection outbreak in several countries [6] [7] [8] .

Therefore, diagnostic tests for coronavirus disease 2019 continue to be in high demand.

Reverse transcription-polymerase chain reaction (RT-PCR) is the standard method for detecting severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) [9] . The steps of RT-PCR are as follows: extract genomic RNA from the virus, reverse transcribe into complementary DNA, amplify the region of interest, and detect fluorescent signals. This method amplifies and detects viral-derived nucleic acids, making it possible to Open Access *Correspondence: hirotsu-bdyu@ych.pref.yamanashi.jp 1 Genome Analysis Center, Yamanashi Central Hospital, 1-1-1 Fujimi, Kofu, Yamanashi, Japan Full list of author information is available at the end of the article sensitively evaluate samples with low viral loads. However, when performed manually, it requires skilled techniques and hands-on time, taking approximately 3-4 h to obtain the results.

Automated devices for testing SARS-CoV-2 have been developed. Random access detection with automated devices enables the rapid return of results to clinicians and patients. In addition, once the sample is loaded into the dedicated reagent, the process from sample to answer is seamless. These tests are straightforward to perform and do not always require skilled personnel. In addition, random access detection can be performed for each sample, making it useful for emergency testing.

We previously showed that the FilmArray Respiratory Panel (v2.1) and Lumipulse antigen test had high accuracy for detecting SARS-CoV-2 in nasopharyngeal swab samples [10] [11] [12] . Next, we planned to strengthen our testing system for SARS-CoV-2 by installing Xpert Xpress SARS-CoV-2. Previous studies reported the performance of Xpert compared with that of the Cobas SARS-CoV-2 assay or in-house real-time RT-PCR assays using clinical samples [13] [14] [15] [16] [17] . However, there have been no reports of a head-to-head comparison between the Xpert and Fil-mArray or Lumipulse. In this study, we aimed to evaluate the assay performance of the Xpert Xpress SARS-CoV-2 and FilmArray Respiratory Panel as automated RT-PCR assays and tested the Lumipulse as an automated antigen quantification test. The same samples were also tested for quantitative RT-PCR (RT-qPCR) as the reference, and the results were compared.

A total of 165 nasopharyngeal swabs were collected and stored in viral transport medium (VTM) (Copan, Murrieta, CA, USA). The patients comprised 78 women and 87 men, with a mean age of 49.2 years (range 1-94). The Institutional Review Board at Yamanashi Central Hospital approved this study and the use of an opt-out consent method (G-2019-1). The requirement for written informed consent was waived because this was an observational study. Patient participation in the study was optional.

We extracted total nucleic acid from 200 μL of VTM using a MagMax Viral/Pathogen Nucleic Acid Isolation Kit (Thermo Fisher Scientific, Waltham, MA, USA) on a KingFisher Duo Prime (Thermo Fisher Scientific) as previously described [18, 19] . These extracted nucleic acids were analyzed by quantitative reverse transcription PCR (RT-qPCR). The residual VTM was used for testing with the Xpert Xpress SARS-CoV-2, FilmArray Respiratory Panel, and Lumipulse SARS-CoV-2 antigen test as described below.

To detect SARS-CoV-2, we performed one-step RT-qPCR in accordance with the protocol developed by the National Institute of Infectious Diseases in Japan [20] . This method amplifies the nucleocapsid (N) gene of SARS-CoV-2 (NC_045512.2) [9] . We made reaction mixture composed of 5 µL of 4 × TaqMan Fast Virus 1-Step Master Mix (Thermo Fisher Scientific), 1.0 μL of 10 μM forward primer, 1.4 μL of 10 μM reverse primer, 0.8 μL of 5 μM probe, 6.8 µL of nuclease-free water, and 5 µL of the nucleic acid sample in a 20-μL total volume as previously described [9] .

The StepOnePlus Real-Time PCR System (Thermo Fisher Scientific) was used for RT-qPCR assays as following condition: 50 °C for 5 min, 95 °C for 20 s, and 45 cycles of 95 °C for 3 s and 60 °C for 30 s. The threshold cycle was examined then the threshold was set at 0.2. We judged the results according to the protocol (version 2.9.1) [20] .

The Xpert Xpress SARS-CoV-2 (Cepheid, Sunnyvale, CA) is an automated real-time RT-PCR test carried out in single-use disposable cartridges. For this assay, 300 µL of VTM was loaded into the sample chamber of the cartridge using a sterile pipette. The cartridges were set in a GeneXpert System (Cepheid) and analyzed. This assay targets the envelope (E) gene and N gene (named N2) and detects SARS-CoV-2 according to the cycle threshold cut-off values.

We performed multiplex real-time RT-PCR assay using the FilmArray Respiratory Panel version 2.1 (bioMérieux, Marcy-l'Etoile, France) [21] . This assay detects 21 pathogens (i.e., SARS-CoV-2 and 17 other viruses and three bacteria) related to respiratory diseases. The kit buffer and 300 µL of VTM were injected into the FilmArray pouch and the reaction proceeded on the FilmArray Torch system (bioMérieux).

The SARS-CoV-2 antigen levels were determined quantitatively with the Lumipulse SARS-CoV-2 antigen test (Fujirebio, Inc., Tokyo, Japan) as previously described [12] . Briefly, 700 μL of the VTM were vortexed and centrifuged at 2000×g for 5 min. Aliquots (100 μL) of the supernatant were tested on the LUMIPULSE G600II system (Fujirebio). For samples with an antigen level > 5000 pg/mL, the samples were diluted with the kit diluent and tested again, and the antigen level was calculated based on the dilution factor. We judged the results according to the manufacturer's instruction.

Sensitivity and specificity values were calculated using the RT-qPCR results as the reference and with the exclusion of inconclusive samples determined by Lumipulse antigen testing. Cohen's kappa (κ) coefficient of results between the two tests with 95% confidence intervals (CI) was calculated. Cohen's κ values > 0.81 were interpreted to indicate near-perfect agreement [22] . Statistical analysis was performed using R Version 3.1.2 (http:// www.rproje ct. org/).

A total of 165 nasopharyngeal swab samples were analyzed to investigate the accuracy of test results. Of the 165 samples, 100 were RT-qPCR positive, and 65 were RT-qPCR negative (Fig. 1A ). Using these samples, Xpert, FilmArray, and Lumipulse tests were conducted.

When the results of RT-qPCR were used as a reference, the overall agreement rate of Xpert was 99.4% (95% CI: 96.7-99.4%) with a sensitivity of 99% (95% CI: 96.8-99%) and specificity of 100% (95% CI: 96.6-100%) (Fig. 1B) . The overall agreement rate of FilmArray was 98.8% (95% CI: 95.9-98.8%) with a sensitivity of 98% (95% CI: 95.6-98%) and specificity of 100% (95% CI: 96.3-100%) (Fig. 1B) .

When inconclusive results (n = 11) were excluded for the Lumipulse test (Fig. 1A) , the overall agreement was 95.5% (95% CI: 91.8-95.5%) with a sensitivity of 92.3% (95% CI: 89.2-92.3%) and specificity of 100% (95% CI: 95.5-100%) (Fig. 1B) . The κ coefficient indicated excellent agreement between each test and RT-qPCR (Xpert vs RT-qPCR, κ = 0.987, 95% CI: 0.932-0.987; FilmArray vs RT-qPCR, κ = 0.975, 95% CI: 0.914-0.975; Lumipulse vs RT-qPCR, κ = 0.909, 95% CI: 0.836-0.909). Compared to Lumipulse, Xpert and FilmArray were equivalent in accuracy to RT-qPCR.

To examine the detectable range of viral loads, the results of three PCR tests (RT-qPCR, Xpert, and FilmArray) were compared. There were 97 samples with positive results determined by all three tests ( Fig. 2 and Table 1 ). Among these 97 positive samples with three tests, the mean viral load was 5.0 log 10 copies/mL (0.2-8.0 log 10 copies/mL), the mean RT-qPCR Ct value was 21.5 (range 12-40), the mean Xpert Ct value (N2) was 23.7 (range 14.2-42.6), and the mean Xpert Ct value (E) was 21.7 (range 12.6-42) ( Table 1) .

Of 100 RT-qPCR positive samples, there were 91 samples with RT-qPCR Ct values < 35 and 9 samples with Ct values ≥ 35 (Fig. 2) . Among 91 samples with Ct < 35, all samples were detectable with both Xpert and FilmArray (Fig. 2) . Among nine samples with Ct ≥ 35, six samples were detected by both Xpert and FilmArray, one sample was not detected by Xpert (the viral load was 0.8 log 10 copies/mL and the RT-qPCR Ct value was 38), and two samples were not detected by FilmArray (viral loads of 1.4 and 0 log 10 copies/mL and RT-qPCR Ct values of 38 and 40, respectively) ( Fig. 2 and Table 2 ). Therefore, all discordant samples had very low viral loads.

The FilmArray is a qualitative test, whereas Xpert targets two different genes, including N and E, and yields Ct values. Thus, we examined the correlation between the Xpert Ct values, RT-qPCR Ct values, and viral loads. The results showed that the Xpert Ct values were highly correlated with the RT-qPCR Ct values for both N2 and E ( Fig. 3A and B ; R 2 = 0.977, Xpert Ct (N2) vs RT-qPCR Ct; R 2 = 0.935, Xpert Ct (E) vs RT-qPCR Ct). In addition, the viral load measured by RT-qPCR and the Xpert Ct value also showed a high correlation ( Fig. 3C and D; R 2 = 0.956, Xpert Ct (N2) vs viral load; R 2 = 0.912, Xpert Ct (E) vs viral load). Because RT-qPCR amplifies the N gene, the data of Xpert N2 were shown to have a higher correlation than E. These results indicate that the Xpert Ct value accurately reflected the viral load in nasopharyngeal swabs.

We also examined the correlation between the antigen levels, viral loads, RT-qPCR Ct values, and Xpert Ct values. As we previously reported [12] , there was a correlation between antigen levels and viral loads (R 2 = 0.893, Fig. 4A ) or RT-qPCR Ct values (R 2 = 0.902, Fig. 4B ). Similarly, there was a correlation between the antigen level and the Xpert Ct values in N2 (R 2 = 0.888, Fig. 4C ) and E regions (R 2 = 0.877, Fig. 4D ).

We evaluated the performance of the newly installed Xpert system in our hospital. The results showed the test performance of Xpert was almost equivalent to that of FilmArray or RT-qPCR. Meanwhile, the Lumipulse antigen test showed a slightly lower detection rate than the NAAT (i.e., RT-qPCR, Xpert and FilmArray).

In particular, the Lumipulse test judged an inconclusive result in a low viral load sample. In this case, the NATT test must be used to determine whether the sample is positive or negative. There was a high correlation between the Xpert Ct value and the RT-qPCR Ct value, viral load and antigen level. Of 165 samples, only one sample (Ct = 38) was discordant between Xpert and RT-qPCR, which was attributed to a very low viral load. Collectively, Xpert yielded accurate test results and quantitatively evaluated the viral load in samples [23] . Therefore, Xpert is considered to be a particularly useful method for random access SARS-CoV-2 testing. Unlike the rapid antigen test (i.e., paper-based assays or lateral flow immunochromatography), the Lumipulse antigen test is based on the chemiluminescence enzyme immunoassay [10] . Compared with the assay performance of the rapid antigen test by other groups [24] [25] [26] [27] [28] [29] , our data show that the Lumipulse antigen test is relatively consistent with the NAAT analysis results among antigen tests [12, 30] . In addition, the Lumipulse antigen test specifically determined a positive result with a high accuracy in specimens with a viral load of more than 100 copies/mL [10] . Our previous analysis showed that NAAT detected the virus for a long time after the onset of symptom, while the Lumipulse antigen test becomes negative [11] . It was often observed that Lumipulse antigen tests turned to be negative after 10 days of symptom onset [12] , which is consistent with the time when infectious viruses are detected in vitro [31] [32] [33] .

The use of random access automated testing is removing barriers to conventional PCR test. The PCR test is the gold standard, but its process is time consuming because a certain number of samples are collected and analyzed in batches. In addition, it requires a skill, which limits its use by personnel who are not familiar with the PCR test. In contrast, Xpert, FilmArray, and Lumipulse take advantage of random access detection, which is useful for emergency. The turnaround time for Xpert, FilmArray, and Lumipulse is within 45 min, 45 min, and 35 min. In Xpert, if the amplification plot reaches the threshold, the test results can be returned in a shorter time. In addition, the simplicity of operation allows for testing to be performed in the absence of experienced personnel. Combining multiple testing methods will improve the speed and efficiency of the testing system in hospitals.

Using different tests that take advantage of on-demand nucleic acid amplification is an effective strategy. Xpert targets a small number of genes (often a single pathogen) due to the limited number of fluorescence wavelengths that can be detected and is inexpensive. In comparison, FilmArray amplifies several targets by multiplex PCR and then performs nested-PCR in multiple wells, which allows for a large number of target genes to be detected simultaneously, but it is more expensive. The Ct value and amplification curve can be confirmed using the Xpert system but not the FilmArray because it is a qualitative test. Therefore, the Xpert may be more advantageous for universal use in testing for SARS-CoV-2 in suspected patients, and the FilmArray may be more useful in patients with underlying medical conditions or children who are at risk of severe illness due to viral infections other than SARS-CoV-2.

In this study, we compared the Xpert Ct values with the RT-qPCR Ct values as a reference and found an extremely high correlation ( Fig. 3A and B) . The correlation was higher for Xpert (N2) than for Xpert (E), likely because RT-qPCR targets the N gene. However, the absolute Ct values of RT-qPCR were closer to those of Xpert (E) than to those of Xpert (N2), except for samples with low viral loads (Tables 1 and 2 ). In some samples with low viral loads, the Xpert Ct (N2), but not the Xpert Ct (E), could be measured as previously reported [23] .

The Ct value is useful as a surrogate indicator of viral load by RT-PCR. However, it should be carefully compared with other datasets because the Ct value changes depending on the following factors. First, it is depended on the target region of PCR amplification. In particular, in the case of in-house assays, it is necessary to carefully consider whether the same conditions are used for primer sequences, reagents, and measuring equipment. Next, the Ct value varies depending on the threshold setting. It is necessary to make sure that the threshold of the assay used is always at a certain level. Therefore, if the amount of virus is to be examined accurately, it is better to measure a standard sample and measure viral load quantitatively from the calibration curve.

Overall, the Xpert, FilmArray, and Lumipulse tests could detect SARS-CoV-2 in samples with sufficient viral loads. The only inconsistency was observed in specimens with low viral loads, but these patients likely release less live virus [31, 32, 34] . In fact, we previously demonstrated that discrepant results between the Lumipulse antigen test and RT-qPCR were attributable to low viral loads collected from seropositive patients [12] . Therefore, the use of random access detection may be effective for patient triage by enabling the rapid return of test results.

In this study, we evaluated the performance of two NAATs (Xpert and FilmArray) and a quantitative antigen test (Lumipulse) to detect SARS-CoV-2. When the result of RT-qPCR was used as reference, the test results of Xpert and FilmArray showed high agreement, indicating that these tests have almost equivalent test accuracy. The Lumipulse antigen test had a slightly lower concordance rate than Xpert and FilmArray, but had high test accuracy except for samples with low viral load. Xpert Ct value correlated with RT-qPCR Ct value, viral load and antigen level, helping to understand the phases of infection. Xpert, FilmArray and Lumipulse tests can be performed automatically on dedicated equipment, achieving seamless and accurate testing for SARS-CoV-2 without the need for complicated procedures.

World Health Organization: Tracking SARS-CoV-2 variants

Centers for Disease Control and Prevention (CDC): Emerging SARS-CoV-2 Variants

European Centre for Disease Prevention and Control: SARS-CoV-2 variants of concern

Discovery of a SARS-CoV-2 variant from the P.1 lineage harboring K417T/E484K/N501Y mutations in Kofu

Detection of R.1 lineage severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) with spike protein W152L/E484K/ G769V mutations in Japan

Nextstrain: real-time tracking of pathogen evolution

SARS-CoV-2 B.1.1.7 lineage rapidly spreads and replaces R.1 lineage in Japan: serial and stationary observation in a community

Discovery of SARS-CoV-2 strain of P.1 lineage harboring K417T/ E484K / N501Y by whole genome sequencing in the city

Double-quencher probes improve detection sensitivity toward Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) in a reverse-transcription polymerase chain reaction (RT-PCR) assay

Comparison of automated SARS-CoV-2 antigen test for COVID-19 infection with quantitative RT-PCR using 313 nasopharyngeal swabs including from 7 serially followed patients

Analysis of a persistent viral shedding patient infected with SARS-CoV-2 by RT-qPCR, FilmArray Respiratory Panel v2.1, and antigen detection

Prospective study of 1,308 nasopharyngeal swabs from 1,033 patients using the LUMIPULSE SARS-CoV-2 antigen test: comparison with RT-qPCR

Test agreement between Roche Cobas 6800 and Cepheid GeneXpert Xpress SARS-CoV-2 assays at high cycle threshold ranges

Brief validation of the novel GeneXpert Xpress SARS-CoV-2 PCR assay

Detection of SARS-CoV-2 by use of the Cepheid Xpert Xpress SARS-CoV-2 and Roche Cobas SARS-CoV-2 assays

Multicenter evaluation of cepheid xpert ® xpress SARS-CoV-2 point-of-care test during the SARS-CoV-2 pandemic

Comparison of commercially available and laboratorydeveloped assays for <i>In Vitro</i> detection of SARS-CoV-2 in clinical laboratories

Environmental cleaning is effective for the eradication of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in contaminated hospital rooms: a patient from the Diamond Princess cruise ship

Pooling RT-PCR test of SARS-CoV-2 for large cohort of 'healthy' and infection-suspected patients: a prospective and consecutive study on 1,000 individuals. medRxiv

Development of genetic diagnostic methods for novel coronavirus 2019 (nCoV-2019) in Japan

Analysis of Covid-19 and non-Covid-19 viruses, including influenza viruses, to determine the influence of intensive preventive measures in Japan

The measurement of observer agreement for categorical data

Multicenter evaluation of the cepheid Xpert Xpress SARS-CoV-2 test

Comparison of two nucleic acid amplification tests (NAATs) and two antigen tests for detection of SARS-CoV-2 from upper respiratory specimens

Evaluation of the Abbott BinaxNOW rapid antigen test for SARS-CoV-2 infection in children: implications for screening in a school setting

Low performance of rapid antigen detection test as frontline testing for COVID-19 diagnosis

Evaluation of rapid antigen test for detection of SARS-CoV-2 virus

Urgent need of rapid tests for SARS CoV-2 antigen detection: evaluation of the SD-Biosensor antigen test for SARS-CoV-2

Field evaluation of a rapid antigen test (Panbio COVID-19 Ag Rapid Test Device) for COVID-19 diagnosis in primary healthcare centers

Comparison of Roche and Lumipulse quantitative SARS-CoV-2 antigen test performance using automated systems for the diagnosis of COVID-19

Virological assessment of hospitalized patients with COVID-2019. Nature. 2020. • fast, convenient online submission • thorough peer review by experienced researchers in your field • rapid publication on acceptance • support for research data, including large and complex data types • gold Open Access which fosters wider collaboration and increased citations maximum visibility for your research: over 100M website views per year submit your research ? Choose BMC

Predicting infectious SARS-CoV-2 from diagnostic samples

SARS-CoV-2 virus culture and subgenomic RNA for respiratory specimens from patients with mild coronavirus disease

Respiratory disease in rhesus macaques inoculated with SARS-CoV-2

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

We thank Hisahiro Tajima and Doura Teruya for technical discussions, all medical and ancillary hospital staff, and the patients for consenting to participate. We thank Melissa Crawford, PhD, from Edanz (https:// jp. edanz. com/ ac) for editing a draft of this manuscript.

Authors' contributions YH: conceptualization, formal analysis, investigation, data visualization, statistical analysis and writing the original draft. MM, MS, YN1, YN2, KH, HS and KA: formal analysis, data collection and investigation. MH and HM: interpretation of data, supervision and project administration. TT and YK: acquisition of data, interpretation of data and sample collection. YM: acquisition of data, interpretation of data, sample collection and project administration. MO: conceptualization, supervision, project administration and review and edit the manuscript. All authors read and approved the final manuscript. YN1 and YN2 correspond to Yume Natori and Yuki Nagakubo, respectively. 

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

The Institutional Review Board at Yamanashi Central Hospital approved this study and the use of an opt-out consent method (G-2019-1). The requirement for written informed consent was waived because this was an observational