key: cord-336465-qrok21qo
authors: Perez, Luis E.; Merrill, Gerald A.; DeLorenzo, Robert A.; Schoenfeld, Thomas W.; Vats, Abhay; Moser, Michael J.
title: Evaluation of the specificity and sensitivity of a potential rapid influenza screening system
date: 2013-01-31
journal: Diagnostic Microbiology and Infectious Disease
DOI: 10.1016/j.diagmicrobio.2012.09.005
sha: 
doc_id: 336465
cord_uid: qrok21qo

Abstract Influenza remains a serious worldwide health threat with numerous deaths attributed to influenza-related complications. It is likely that transmission of influenza and both the morbidity and mortality of influenza could be reduced if inexpensive but reliable influenza screening assays were more available to the general public or local medical treatment facilities. This report provides the initial evaluation of a pilot system designed by Lucigen Corp. (Middleton, WI, USA) as a potential rapid near point-of-care screening system for influenza A and influenza B. The evaluation of specificity and sensitivity was conducted on stored nasal swab samples collected from emergency department patients presenting with influenza-like symptoms at a large military academic hospital and on de-identified nasal swabs and isolated RNA from a local epidemiology laboratory. The gold standard for assessment of specificity and sensitivity was the Luminex® Respiratory Viral Panel.

Seasonal influenza epidemics result in about 3-5 million cases and around 0.25-0.5 million deaths each year worldwide (WHO, 2009 ). In the United States, up to 300,000 people are hospitalized annually and 10,000-40,000 die from influenza-related complications (CDC, 1999) . Diagnosis of influenza A and B by clinical manifestations alone is difficult because of overlapping symptoms from a variety of pathogens including rhinovirus (Arden and Mackay, 2010) , coronavirus (Renois et al., 2010) , parainfluenza virus (Lau et al., 2005) , respiratory syncytial virus (Freymuth et al., 2004) , adenovirus (Lina et al., 1996) , metapneumovirus (Debur et al., 2010) , enterovirus (Lina et al., 1996) , and Streptococcus pyogenes (Yamada et al., 2010) . Use of antibiotics to treat upper respiratory tract infections without proper diagnosis is a common practice in the United States (Franck and Smith, 2010; Linder et al., 2003) . For instance, in a national ambulatory network study of 52,135 upper respiratory tract episodes identified, 65% received antibiotics (Gill et al., 2006) , although respiratory viruses are responsible for approximately 80% of respiratory infection (Mahony, 2008) . Use of antibiotics, which are of little benefit for viral infections, can play a major role in the development and spread of antibioticresistant microorganisms and substantially increase health care costs. Thus, development of a rapid, cost-effective, and accurate screening system for influenza virus available to the general population and/or local physicians and medical treatment facilities would likely reduce the misuse of antibiotic therapy for treatment of influenza infections. Rapid identification of influenza infections would further reduce transmission of virus including nosocomial infections, allow for timely antiviral therapy, and likely reduce the morbidity and mortality of influenza-related complications thereby lowering health care costs.

Currently, there are several methods used to diagnose influenza A and B including viral culture, lateral flow immunoassays (LFIAs), direct fluorescent antibody tests (DFAs), and nucleic acid tests (NATs). Diagnosis of influenza A or B through viral culture is effective but is labor intensive, time consuming, and may require highly trained personnel and several days to provide confirmed results (Balada-Llasat et al., 2011; Ruest et al., 2003) . Current LFIA methods for influenza A or B have high specificity (90-100%), are cost effective, and results can be acquired within 15 to 30 min, but, unfortunately, these tests have been shown to exhibit variable sensitivity ranging from as low as 10% up to 100% (Louie et al., 2009) . DFA influenza assays demonstrate sensitivities of 60% to 80%, and results can be obtained within 2 h, but require high level of technical proficiency to perform and culture to confirm the results (Cram et al., 1999) . NATs offer high sensitivity and specificity for the diagnosis of influenza A or B (Jenny et al., 2010; Louie et al., 2009; Pabbaraju et al., 2008) . However, use of NATs in the clinical sector may be prohibited because of high cost and delayed reporting of results as compared to LFIA and DFA. The lack of an affordable, near point-of-care, sensitive, and specific screening system for influenza virus created a void that was addressed by Lucigen Corp. (Middleton, WI, USA) by designing a first-generation system to test the feasibility of a nucleic acid lateral flow system for testing both influenza A and B.

The PyroScript® influenza A and B reagents require the use of a nucleic acid lateral flow (NALF) device for the detection of influenza A or B-amplified RNA molecules. NALF devices work in a manner analogous to lateral flow immunoassays. A visual result can be obtained within 2 h after sample collection allowing rapid interpretation of results without the need of complex or expensive instruments. The PyroScript® influenza screening reagents could be the alternative solution for a rapid influenza A and B test in the clinical sector.

This research was conducted under an internal review boardapproved protocol which allowed for the collection of clinical samples from patients presenting with influenza-like respiratory symptoms in the Emergency Department (ED) of Brooke Army Medical Center after informed consent was obtained. Transfer of de-identified specimens (stored virus or, in the case of H1N1 novel 2009 [swine flu] specimens, isolated viral RNA) from a local epidemiology reference laboratory was also approved. Twenty nasal swab samples collected from ED patients with influenza-like symptoms, 5 archived de-identified Streptococcus pyogenes bacterial specimens, and 75 deidentified specimens transferred from the local epidemiology laboratory were investigated. Although the viral samples obtained from the epidemiology reference laboratory were de-identified (all personal health information and personal identifiers removed), they were supplied with the etiologic agent of each sample identified. The laboratory personnel performing the extractions, Luminex RVP assay, and Lucigen PyroScript tests were blinded to the clinical results until the conclusion of all testing. These samples included 24 novel influenza A H1N1 RNA specimens and 12 influenza A H1N1 seasonal, 12 influenza A H3N2 seasonal, 15 influenza B seasonal; 5 adenovirus, 4 parainfluenza 3 viral specimens, and 3 negative samples (no virus detected). The 5 Streptococcus pyogenes culture-positive throat swab samples were confirmed via polymerase chain reaction (PCR) with specific primers for the ubiquitous proS gene as described before (Livezey et al., 2011) and were included in the study as influenzanegative controls. All samples were stored at −80°C.

The objective of this preliminary study was to assess the specificity and sensitivity of the influenza A and B NALF systems as a potential rapid screening assay. This system, because of its simplicity and independence from expensive or complicated instrumentation, could be used in near point-of-care settings. Analysis time was considered, but did not include sample preparation time since our samples were stored specimens. For our convenience and reproducibility purposes, as well as for comparison to the Food and Drug Administration approved xTAG® (Luminex Molecular Diagnostics, Toronto, Ontario, Canada) Respiratory Viral Panel (RVP) test, nucleic acid isolation was performed by an automated system (NucliSENS® easyMAG® system, bioMérieux, Marcy l'Etoile, France) as described in Bolotin et al. (2009) . The purified nucleic acids obtained were stored at −80°C.

The gold standard for the specificity and sensitivity assessment was the results of the FDA approved RVP, a nucleic acid test. The RVP test had a sensitivity of 98.2% and a specificity of 96.4% in a study of 247 clinical samples when compared to DFA and culture results (Mahony et al., 2007) . All viral isolates were amplified, hybridized, and detected with the Luminex® IS-200 instrument (Luminex, Austin, TX) using the xTAG RVP kit following the vendor's protocol. Briefly, 5 μL of purified nucleic acid from each sample was reverse transcribed in a 25-μL multiplex reverse transcription polymerase chain reaction (RT-PCR) tube. A multiplex target specific primer extension (TSPE) reaction was prepared per sample using 5 μL of treated RT-PCR. After TSPE, 3.5 μL of reaction product of each reaction was added directly to a micro-well containing 20 μL of RVP Luminex bead mixture for hybridization. Phycoerythrin reporter (100 μL) was added to each well containing hybridization product to identify the presence of a virus by assessment of the fluorescence in a Luminex IS-200. Results generated were analyzed using the software component of the kit (TDAS RVP-I). In addition, all novel H1N1 2009 RNA samples were assayed by a microarray technique (ElectraSense® assay, CombiMatrix, Corp., Mukilteo, WA, USA) to positively identify the H1(sw)N1 variant (Straight et al., 2010) .

The PyroScript® Influenza reagents (Lucigen) were tested for all samples in this study. One reaction per test was prepared on ice in a 200-μL PCR tube as follows: 6.25 μL nuclease-free water, 12.50 μL PyroScript Isothermal 2× master mix, 1.25 μL 20× Primer Mix, and 5 μL of nucleic acid sample. The reaction was mixed and centrifuged briefly before incubating for 40 min at 72°C in a thermal cycler (Eppendorf 5417R, Hamburg, Germany) preheated to 72°C. The reaction was stopped at 4°C on ice until the 200-μL PCR tube containing specific amplicons was loaded in a cartridge of a Type I BESt™ Cassette (BioHelix Corporation, Beverly, MA, USA) for detection according to the manufacturer's instructions. The influenza A or B-amplified molecules entered the DNA strips via capillary action. When the cassette was closed, the PCR tube released the 6-carboxy fluorescein (FAM) and biotin-labeled amplicons (flu A or flu B reagent) and detected via gold nanoparticle-labeled anti-FAM antibodies. Results were visually read after 10-15 min, photographed, and results recorded.

Each ED sample was routinely tested using a rapid antibody assay (BinaxNOW, Alere, Waltham MA, USA), DFA, and viral culture by the BAMC clinical laboratory (data not shown). All seasonal influenza A samples were previously identified by the local epidemiology laboratory as influenza A H1N1 and H3N2 subtypes, respectively, as per their standard protocol (data not shown). Influenza B samples were similarly identified by the epidemiology laboratory. All 12 influenza A H1N1 seasonal viral samples were confirmed by the Luminex RVP assay as influenza A H1. Similarly, all 12 influenza A H3N2 seasonal viral isolates were confirmed as influenza A H3 by the RVP assay. Two additional samples collected at the BAMC ED were identified by the RVP as influenza A (no-subtype) and were classified as novel H1N1 influenza A, although this was not confirmed by PCR or microarray. Six of the 20 ED samples were identified by the Luminex RVP assay as positive for virus: 2 influenza A no-subtype (as discussed above), 2 rhinovirus, 1 parainfluenza 3, and 1 metapneumovirus. In addition, all 15 influenza B, 5 adenovirus, and 4 parainfluenza 3 samples from the epidemiology laboratory were also confirmed by the Luminex RVP assay. Fourteen of the 20 ED samples, plus the 3 samples from the epidemiology laboratory with no virus detected, and the 5 Streptococcus pyogenes bacterial samples were used as the 22 negative controls for both the influenza A and influenza B calculations (see Table 1 A total of 3 false-positive (positive by the Lucigen system but negative by the gold standard Luminex RVP assay) influenza A and 6 false-positive influenza B results were observed with the PyroScript influenza tests. In the case of influenza A, all samples not identified by the RVP assay as influenza A are considered as true negatives for calculation of the influenza A specificity. Conversely, all samples not identified by the RVP assay as influenza B are considered as true influenza B negatives. Thus, the specificity of the PyroScript influenza reagents are 50/53 (94.3%) for influenza A and 84/90 (93.3%) for influenza B tests.

The recent 2009 influenza pandemic caused about 55 million human cases of novel 2009 influenza H1N1 with approximately 246,000 H1N1-related hospitalizations and about 11,160 H1N1related deaths in the United States alone from April to December of 2009 according to estimates from the Centers for Disease Control and Prevention (CDC, 2010) . The development of easy-to-use, costeffective, and reliable new diagnostic methods accessible to primary health care providers for screening patients is needed for the rapid detection of influenza A and influenza B. Rapid screening for influenza at primary health care facilities would aid in sequestering infected patients from the general population to limit transmission and would reduce the unnecessary use of antibiotics provided to influenza patients misdiagnosed as having bacterial respiratory illnesses. The PyroScript influenza reagents in combination with the cartridge of Type I BESt™ Cassette tested in our study demonstrated excellent outcome parameters with greater than 90% specificity and sensitivity for both influenza A and B tests. Similar isothermal amplification tests in combination with Type I BESt™ Cassette have been developed and used to rapidly detect other pathogenic microorganisms including herpes simplex virus types 1 and 2 (Kim et al., 2011) , Mycobacterium tuberculosis (Motre et al., 2011) , human immunodeficiency virus (Tang et al., 2010) , and toxigenic Clostridium difficile (Chow et al., 2008) .

The 24 novel influenza A H1N1 samples were initially identified by the RVP at the epidemiology laboratory as influenza A no-subtype. Novel influenza A H1N1 is detected with the RVP assay as "influenza A no-subtype" due to recognition of the matrix gene, combined with lack of detection of either the seasonal H3 or the H1 target genes. Investigators have reported that during the 2009 influenza season all "influenza A no-subtype" cases detected with the RPV test were most likely novel influenza A H1N1 (Ginocchio and St George, 2009; Vinikoor et al., 2009) . All 24 samples were confirmed via RT-PCR as novel influenza A H1N1 (data not shown) prior to being received at BAMC. Of these 24 RNA samples tested at BAMC by the Electra-Sense®, 23 were confirmed as influenza A H1(sw)N1 variants (Straight et al., 2010) . The novel influenza A H1(sw)N1 RNA sample not detected by this microarray technique was also reevaluated by the Luminex RVP assay, and no viral etiology could be identified. Therefore this sample was eliminated from this study.

Extraction of nucleic acids for the detection of influenza virus with automated robotic systems has been used by other investigators (Belongia et al., 2010; Bolotin et al., 2009; Bose et al., 2009; Yang et al., 2011) , as was done in this study using the NucliSENS® easyMAG® system. The easyMAG® system requires at least 1 h to extract nucleic acids from 24 clinical samples including sample preparation time and 10-min viral/bacterial lysis time. Automated nucleic acid extraction systems are excellent instruments for use in reference laboratories where a larger number of samples are processed but are bulky and expensive to use in small laboratories or physician's offices. Alternative nucleic acid extraction methods must be incorporated into future PyroScript influenza screening systems to fulfill the time efficiency requirement of a rapid screening assay. In addition, the requirement to screen for influenza A and B separately with the tested PyroScript influenza system requires 2 reactions with different reagents and 2 detection cassettes. Ideal future versions of the PyroScript influenza screening system should also allow for simultaneous screening of both viruses and might include other respiratory virus targets.

The objective of this study was to determine the sensitivity and specificity of the PyroScript Influenza system. Loss of sensitivity occurs if the test assay does not produce a positive result when the gold standard assay is determined to be positive. The 91.8% influenza A and 93.3% influenza B sensitivity resulted from failure to identify 5 RVP-positive samples as positive by the PyroScript assay. There could be several reasons for the PyroScript not being able to identify some of the true positives. One such possibility could be due to stored RNA deterioration with time. Since insufficient sample remained for additional retesting by the RVP assay, the possibility cannot be excluded that the 3 H1N1 novel, the seasonal influenza A, and the seasonal influenza B samples that appear as false negatives could be from long-term storage-induced RNA degradation and/or freeze-thaw cycles (Botling et al., 2009; Ginocchio et al., 1997) . Loss of specificity occurs due to a false-positive result (positive by the PyroScript assay but not the RVP). In this study, a total of 9 false positives were observed. One possibility for these false-positive results could be that the primers for RVP and PyroScript assay are not identical. An additional possibility is that there may have been a mutation in the viral genome where the RVP primers/probes bind which did not affect the isothermal amplification employed in the PyroScript tests. Another possibility could be nonspecific amplification by the PyroScript assay.

In conclusion, the PyroScript influenza reagents in combination with the cartridge of Type I BESt™ Cassette are self-contained disposable detection devices and do not require special instrumentation or complicated specimen handling to obtain a visual result within 2 h after nucleic acid isolation. As such, the PyroScript influenza A and B tested in this study show the potential to fill the screening void for rapid influenza diagnosis. Future development of the PyroScript influenza reagents, screening system, and the implementation of alternative nucleic acid extraction methods to provide a true point-of-care NAT for human influenza diagnostics is warranted.

Newly identified human rhinoviruses: molecular methods heat up the cold viruses

Evaluation of commercial ResPlex II v2.0, MultiCode-PLx, and xTAG respiratory viral panels for the diagnosis of respiratory viral infections in adults

Clinical characteristics and 30-day outcomes for influenza A 2009 (H1N1)

Validation of the TaqMan Influenza A detection kit and a rapid automated total nucleic acid extraction method to detect influenza A virus in nasopharyngeal specimens

Rapid semiautomated subtyping of influenza virus species during the 2009 swine origin influenza A H1N1 virus epidemic in

Impact of thawing on RNA integrity and gene expression analysis in fresh frozen tissue

Centers for Disease Control and Prevention (CDC). CDC Estimates of 2009 H1N1 influenza cases, hospitalizations and deaths in the United States

Application of isothermal helicase-dependent amplification with a disposable detection device in a simple sensitive stool test for toxigenic Clostridium difficile

Diagnostic testing for influenza: review of current status and implications of newer treatment options

Impact of human metapneumovirus infection on in and outpatients for the years 2006-2008 in Southern Brazil

Antibiotic use for acute upper respiratory tract infections in a veteran population

Epidemiology and diagnosis of respiratory syncytial virus in adults

Use of antibiotics for adult upper respiratory infections in outpatient settings: a national ambulatory network study

Likelihood that an unsubtypeable influenza A virus result obtained with the Luminex xTAG respiratory virus panel is indicative of infection with novel A/H1N1 (swine-like) influenza virus

Effects of specimen collection, processing, and storage conditions on stability of human immunodeficiency virus type 1 RNA levels in plasma

Evaluation of the Xpert Flu A Panel nucleic acid amplification-based point-of-care test for influenza A virus detection and pandemic H1 subtyping

A rapid and simple isothermal nucleic acid amplification test for detection of herpes simplex virus types 1 and 2

Human parainfluenza virus 4 outbreak and the role of diagnostic tests

Surveillance of communityacquired viral infections due to respiratory viruses in Rhone-Alpes (France) during winter 1994 to 1995

Association between antibiotic prescribing and visit duration in adults with upper respiratory tract infections

Analysis of group A Streptococcus gene expression in humans with pharyngitis using a microarray

Point-of-care testing for pandemic influenza and biothreats

Development of a respiratory virus panel test for detection of twenty human respiratory viruses by use of multiplex PCR and a fluid microbead-based assay

Detection of respiratory viruses by molecular methods

Improving isothermal DNA amplification speed for the rapid detection of Mycobacterium tuberculosis

Comparison of the Luminex xTAG respiratory viral panel with in-house nucleic acid amplification tests for diagnosis of respiratory virus infections

Rapid detection of respiratory tract viral infections and coinfections in patients with influenza-like illnesses by use of reverse transcription-PCR DNA microarray systems

Comparison of the Directigen Flu A+B test, the QuickVue influenza test, and clinical case definition to viral culture and reverse transcription-PCR for rapid diagnosis of influenza virus infection

A novel electrochemical device to differentiate pandemic (H1N1) 2009 from seasonal influenza

Nucleic acid assay system for tier II laboratories and moderately complex clinics to detect HIV in low-resource settings

Influenza A virus subtyping: paradigm shift in influenza diagnosis

Influenza (seasonal), fact sheet no

Invasive group A streptococcal infection in pregnancy

Comparison of commercial systems for extraction of nucleic acids from DNA/RNA respiratory pathogens

The authors would like to acknowledge the help of Dr. John A. Ward with the statistical consultations he provided and Suzanne McCall for her laboratory and technical support.