key: cord-0709212-56fd4cqc authors: Cobb, Beth L; Lloyd, Matthew; Hock, Karl G; Farnsworth, Christopher W title: Clinical Performance of a Lateral Flow SARS-CoV-2 Total Antibody Assay date: 2022-03-17 journal: J Appl Lab Med DOI: 10.1093/jalm/jfac003 sha: 3cc05a8b03f850b624ffcd2748de6bd12ac435af doc_id: 709212 cord_uid: 56fd4cqc BACKGROUND: Serological assays for SARS-CoV-2 are important tools for diagnosis in patients with negative RT-PCR testing, pediatric patients with multisystem inflammatory syndrome, and serosurveillance studies. However, lateral flow-based serological assays have previously demonstrated poor analytical and clinical performance, limiting their utility. METHODS: We assessed the ADEXUSDx COVID-19 lateral flow assay for agreement with diagnostic RT-PCR testing using 120 specimens from RT-PCR–positive patients, 77 specimens from symptomatic RT-PCR–negative patients, and 47 specimens obtained prepandemic. Specimens collected <14 days from symptom onset in RT-PCR–positive patients were compared relative to the Abbott SARS-CoV-2 IgG assay. RESULTS: The ADEXUSDx COVID-19 Test yielded an overall positive percent agreement (PPA) of 92.5% (95%CI 85.8 to 96.3) and negative percent agreement of 99.2% (95% CI 94.9–100.0) relative to RT-PCR and in prepandemic specimens. Relative to days from symptom onset, the PPA after 13 days was 100% (95% CI 94.2–100); from 7 to 13 days, 89.7 (95% CI 71.5–97.2); and from 0 to 7 days, 53.8 (95% CI 26.1–79.6). The overall agreement between the Abbott and ADEXUSDx assays was 80.9%. Twenty-five specimens were positive by both assays, 9 specimens were negative by both assays, and 8 specimens were positive by only the ADEXUSDx assay. CONCLUSIONS: We demonstrate high PPA and negative percent agreement of the ADEXUSDx COVID-19 assay and diagnostic testing by RT-PCR, with PPA approximately 90% by 7 days following symptom onset. The use of waived testing for antibodies to SARS-CoV-2 with high sensitivity and specificity provide a further tool for combatting the COVID-19 pandemic. According to the Infectious Diseases Society of America, serological testing for SARS-CoV-2 is useful for conducting serosurveillance studies, for the assessment of multisystem inflammatory syndrome in children, and for evaluating patients with high suspicion of COVID-19 but persistently negative molecular testing, (1, 2) . While there are more serological assays available for SARS-CoV-2 than any other infectious disease, until recently few waived methods existed with emergency use authorization (EUA) (3) . As a result, most of the serological testing for COVID-19 requires trained phlebotomists performing blood draws, often a hindrance to enrolling participants in studies and frequently a challenge in pediatric patients (4) . Furthermore, the emergence of the COVID-19 pandemic led to the introduction of hundreds of lateral flow-based, sample to answer assays from companies with limited experience in the in vitro diagnostics market (5, 6) . Most of these devices had remarkably poor clinical sensitivity and specificity (7, 8) , leading the US Food and Drug Administration (FDA) to require EUA for all serological testing for SARS-CoV-2. While considerable literature has assessed fully automated methods for SARS-CoV-2 serological testing and has found those with EUA to be suitable for detecting patients with previous SARS-CoV-2 infection (2, (9) (10) (11) (12) , little exists in the published literature assessing the clinical performance of EUA SARS-CoV-2 serological assays designated as waived by the FDA. The ADEXUSDx COVID-19 Test received EUA in May 2021 for qualitative detection of total antibodies to SARS-CoV-2 in human venous whole blood, plasma, serum, and fingerstick whole blood. The purpose of this study was to perform a clinical evaluation of the ADEXUSDx COVID-19 Test using RT-PCR as the gold standard for diagnosis of SARS-CoV-2 infection. This study was approved by the Washington University Institutional Review Board. All specimens were clinical remnants collected in EDTA plasma tubes. One hundred twenty specimens were from patients presenting symptomatic and confirmed positive for COVID-19 by EUA Cepheid Xpert Xpress SARS-CoV-2 test, 77 specimens from patients confirmed negative by EUA Cepheid Xpert Xpress SARS-CoV-2 test and clinically adjudicated as non-COVID-19 patients, and 47 prepandemic specimens were used. All specimens were frozen and stored at À80 C prior to analysis. Each specimen underwent a single freeze-thaw cycle prior to testing. All postpandemic specimens were collected from April through August 2020. The confirmed positive specimens were selected according to range of days following positive RT-PCR test; 30 specimens were selected 0 to 6 days post-RT-PCR; 30 specimens, 7 to 13 days post-RT-PCR; 30 specimens, 14 to 20 days post-RT-PCR; and 30 specimens, >20 days post-RT-PCR. Time from symptom onset was adjudicated by 2 independent reviewers using physician encounter notes in the electronic medical record (EPIC, Epic Systems Verona). All patients in this cohort were hospitalized due to symptoms from COVID-19. Age and sex were also collected but were not available for prepandemic specimens included in the validation study. Specimens were tested using ADEXUSDx COVID-19 Tests developed by NOWDiagnostics. High volume assays for assessing serological response to SARS-CoV-2 have been extensively reported on, but little exists in the published literature assessing lateral flow-based assays designated as waived under an emergency use authorization. Here we describe the performance of the ADEXUSDx COVID-19 Test and demonstrate high positive and negative percent agreement relative to SARS-CoV-2 RT-PCR testing, with better agreement than a high throughput automated method <14 days from symptom onset. The use of waived testing for antibodies to SARS-CoV-2 with high sensitivity and specificity provide a further tool for combatting the COVID-19 pandemic. The assay uses a lateral flow platform that is engineered for separation of plasma from whole blood. The sample first passes through the plasma separating membrane. The membrane is impregnated with SARS-CoV-2 recombinant receptor binding domain (RBD) of the viral spike protein and rabbit IgG, both conjugated with colloidal gold. Antibodies specific to SARS-CoV-2 in the specimen bind to the gold-labeled SARS-CoV-2 recombinant antigen, and the complex is captured by immobilized SARS-CoV-2 antigen with the appearance of a visible test line indicating a detectable level of SARS-CoV-2 antibody. Rabbit IgG binds to the immobilized polyclonal anti-rabbit antibody at the control line. Test (T) and control (C) lines on each cassette are visually read. This test is approved by the FDA under an EUA for venous whole blood and plasma testing in a moderate complexity laboratory or capillary whole blood as a waived complexity test. The reported positive agreement in plasma relative to RT-PCR confirmed cases of SARS-CoV-2 is 100% (95% CI 82.4-100) 15 days after diagnostic testing and the negative percent agreement is 97.9% (95% CI 95.2-99.1) (13). All testing was performed by a trained technologist according to the laboratory pipette test method of the manufacturer's reference instructions. Briefly, 40 uL of plasma was applied to the sample application zone of the cassette. Specimens from patients with RT-PCR confirmed infection, but specimens <14 days from symptom onset were also tested by the Abbott SARS-CoV-2 IgG assay. This assay is a chemiluminescent microparticle immunoassay to target antibodies to the SARS-CoV-2 nucleocapsid protein. The assay reports positive results as a signal to calibrator ratio, with a signal !1.4 considered positive. Diagnostic positive percent agreement (PPA) and negative percent agreement (NPA) for each assay were calculated using RT-PCR testing as the gold standard. CIs were calculated according to the efficient score method as previously described (14) . All statistics were performed using GraphPad Prism v9. Positive specimens (n ¼ 120) were acquired from 45 subjects. The average age of the test subjects was 64.8 years (range 40-90), and 25/45 (57.8%) subjects were male. The ADEXUSDx COVID-19 Test yielded an overall PPA of 92.5% (95% CI 85.8-96.3) ( Table 1 ). The highest agreement was found after 13 days from RT-PCR-positive result (PPA 100% for 14-20 days and þ21 days). Relative to days from symptom onset, the PPA after 13 days was 100% (95% CI 94.2-100). The PPA from 7 to 13 days from symptom onset was 89.7 (95% CI 71.5-97.2), and from 0 to 7 days, it was 53.8 (95% CI 26.1-79.6). Thirteen days from symptom onset was the latest timepoint in which a specimen from a patient with RT-PCR confirmed SARS-CoV-2 infection was negative by the ADEXUSDx COVID-19 Test (Fig. 1) . The overall NPA of the ADEXUSDx COVID-19 Test was 99.2% (95% CI 94.9-100.0) relative to diagnostic, RT-PCR testing for SARS-CoV-2, and prepandemic specimens. The NPA was 100.0% (95% CI 94.1-100.0) and 97.9% (95% CI 87.3-99.9), respectively, in specimens from symptomatic but SARS-CoV-2 RT-PCR-negative patients and specimens drawn prepandemic. All specimens from patients that were RT-PCRpositive but <14 days post-COVID-19 symptom onset (n ¼ 42) were also tested using the Abbott Architect SARS-CoV-2 IgG assay. This assay yielded a PPA of 30.8% (95% CI 10.4-61.1) <7 days postsymptom onset and 72.4% (95% CI 52.5-86.6) from 7 to 13 days post-symptom onset (Fig. 2) relative to RT-PCR testing. The overall agreement between the Abbott and ADEXUSDx assays was 80.9% (kappa ¼ 0.57, 95% CI 0.31-0.84). Twentyfive specimens were positive by both assays, 9 specimens were negative by both assays, and 8 specimens were positive by only the ADEXUSDx assay. No specimens were positive by the Abbott assay but negative by the ADEXUSDx assay. Waived assays with EUA or FDA approval and high sensitivity/specificity are important tools for serosurveillance studies and for the diagnosis of multisystem inflammatory syndrome. However, little is available in the published literature assessing the clinical performance of these devices. In this study, we demonstrate high PPA and NPA of the ADEXUSDx COVID-19 lateral flow assay in plasma from patients relative to diagnostic testing for SARS-CoV-2. Under the FDA EUA, serological assays for SARS-CoV-2 must meet a clinical PPA with molecular testing of 90% and a NPA of 95% (15) . Interestingly, in hospitalized patient cohorts, many of these assays have frequently not performed as well as package inserts claimed by manufacturers (9, 10) . This is likely due to multiple comorbidities of hospitalized patients. Nonetheless, in this study we found a PPA of >100% after 13 days from symptom onset and a PPA approaching 90% from days 7 to 13 in a hospitalized patient cohort. Importantly, serological testing has not been considered diagnostic, in part because of the low sensitivity at early timepoints from symptom onset (16, 17) . At our hospital, the average time from symptom onset to emergency department presentation is approximately 3 days, but approximately 26% of patients present by 7 days or later after symptom onset (unpublished data). While Infectious Diseases Society of America guidelines and the CDC do not recommend serological testing until 2 weeks after symptom onset (2, 17), higher sensitivity assays like those described here may be useful at earlier timepoints after symptom onset in symptomatic patients that are persistently RT-PCR-negative. However, more studies As with all serologic assays for SARS-CoV-2, there is a trade-off of enhanced analytic sensitivity with reduced analytic specificity. However, the NPA of the ADEXUSDx COVID-19 Test was also quite high, exceeding 99% across all samples. Interestingly, the specimen that was positive was a prepandemic (2018) specimen drawn from a healthy control. Unfortunately, no demographic information was available for prepandemic specimens. However, the specificity found in this study is similar to that reported in the manufacturer's instruction for use (97.9% in serum or plasma). This does imply a potentially important role of this assay for serosurveillance, particularly when combined with other data such as patient demographics to help guide public health efforts. Previous studies have demonstrated relatively poor performance of lateral flow based SARS-CoV-2 serological assays (7, 8) . In contrast, we demonstrate comparable performance of the ADEXUSDx COVID-19 assay with a high throughput EUA-approved serological assay <14 days from symptom onset. Of note, we observed that specimens from patients <14 days from symptom onset had a higher PPA than the Abbott Architect SARS-CoV-2 IgG assay with diagnostic RT-PCR testing. This may be due in part to the different targets of the assays: the Abbott assay targets the SARS-CoV-2 nucleocapsid protein, and the ADEXUSDx targets the RBD domain of the viral spike protein. The assay design of the ADEXUSDx also detects both IgG and IgM, which may allow for earlier detection of SARS-CoV-2 infection than assays that only detect IgG. However, reports on this with other total Ig assays are conflicting (10) . Finally, several immunosuppressed patient populations have been shown to have reduced or no measurable antibody response following vaccination with SARS-CoV-2 (18) . While antibody testing was not recommended for assessing immunity by the FDA in May 2021 when SARS-CoV-2 vaccines were being widely distributed across the United States (19) , these guidelines have continued to change over time. Recently published studies have found the benefit of a booster vaccination dose in immunosuppressed patients (20) . Therefore, waived testing for assessment of antibodies to RBD could provide means for distinguishing at-risk patients and administering a booster within a single visit to a physician's office or clinic. However, further studies are required to determine the utility of this approach and the necessity of antibody testing for distinguishing at-risk patients. Interestingly, the RECOVERY study recently demonstrated a reduction in mortality in patients who were seronegative and treated with a combination of monoclonal antibodies targeting the RBD domain of the viral spike protein but not patients who were seropositive (21) . Therefore, near-patient serologic testing may be useful for rapidly assessing patient populations that will benefit from certain therapies, but further studies are needed. There were several limitations of this study. The use of a single observer for all testing did reduce inconsistencies, but real-world performance may vary for qualitative, lateral flow-based methods. Importantly, this study was not designed to assess the clinical sensitivity and specificity of the ADEXUSDx assay in capillary blood. It is possible that variable performance would be observed in this population given potential complexities of collecting capillary specimens. Nonetheless, studies submitted to the FDA for EUA demonstrated comparable performance between plasma and capillary blood, supporting the use of this device for each specimen matrix. It is possible that patients in the negative group represented false negatives, conflating the PPA and NPA of the ADEXUSDx assay. However, this is unlikely given that no patients negative by RT-PCR were clinically adjudicated as having COVID-19. Furthermore, this study was not designed to assess cross-reactivity with antibodies to seasonal coronaviruses, HIV, hepatitis C, or influenza. Studies in the manufacturer's instruction for use document do imply that routine crossreactivity is not expected (13) . Nonetheless, further independent studies are required to assess this. Another important limitation was that lot-tolot variability was not assessed in this study. Previous studies with lateral flow-based methods from other manufacturers have demonstrated poor agreement between lots of cartridges (22) . Independent validation of the variability of ADEXUSDx device lots are needed in follow-up studies. Finally, this study used specimens from an entirely hospitalized cohort of adults. Further studies are required to assess the clinical utility in pediatric patient populations and in the outpatient setting. In conclusion, we demonstrate very high PPA and NPA of the ADEXUSDx COVID-19 assay and diagnostic testing by RT-PCR on the Cepheid XPRESS SARS-CoV-2 assay, with PPA approximately 90% by 7 days following symptom onset. The use of waived testing for antibodies to SARS-CoV-2 with high analytical sensitivity and specificity provide a further tool for combating the SARS-CoV-2 pandemic. Author Contributions: All authors confirmed they have contributed to the intellectual content of this paper and have met the following 4 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; (c) final approval of the published article; and (d) agreement to be accountable for all aspects of the article thus ensuring that questions related to the accuracy or integrity of any part of the article are appropriately investigated and resolved. Farnsworth received research support from NOWDiagnostics, Abbott Laboratories, Beckman Coulter, Roche Diagnostics, and Siemens Healthineers. Expert Testimony: None declared. Patents: None declared. The funding organizations played a direct role in the design of study and preparation of manuscript. The funding organizations played no role in the choice of enrolled patients, review and interpretation of data, or final approval of manuscript. Infectious Diseases Society of America guidelines on the treatment and management of patients with COVID-19 IDSA guidelines on the diagnosis of COVID-19: serologic testing In vitro diagnostics EUAs-serology and other adaptive immune response tests for SARS-CoV-2 Overcoming phlebotomy challenges in pediatric patients SARS-CoV-2 serology: much hype, little data The role of antibody testing for SARS-CoV-2: is there one? Evaluation of SARS-CoV-2 serology assays reveals a range of test performance Comparative assessment of multiple COVID-19 serological technologies supports continued evaluation of point-of-care lateral flow assays in hospital and community healthcare settings Clinical performance of two SARS-CoV-2 serologic assays Clinical performance of the Roche SARS-CoV-2 serologic assay Association between SARS-CoV-2 neutralizing antibodies and commercial serological assays Performance characteristics of four high-throughput immunoassays for detection of IgG antibodies against SARS-CoV-2 Food and Drug Administration Two-sided confidence intervals for the single proportion: comparison of seven methods Serology template for test developers Important information on the use of serological (antibody) tests for COVID-19-letter to health care providers CDC. Interim guidelines for COVID-19 antibody testing. Last updated Antibody response to SARS-CoV-2 vaccines in patients with hematologic malignancies FDA advises against use of SARS-CoV-2 antibody test results to evaluate immunity or protection from COVID-19, including after vaccination Randomized trial of a third dose of mRNA-1273 vaccine in transplant recipient RECOVERY Collaborative Group. Casirivimab and imdevimab in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial Evaluation of production lots of a rapid point-of-care lateral flow serological test intended for identification of IgM and IgG against the N-terminal part of the spike protein (S1) of SARS-CoV-2