key: cord-322641-mz0b91xr
authors: Farnsworth, Christopher W; Anderson, Neil W
title: SARS-CoV-2 Serology: Much Hype, Little Data
date: 2020-04-28
journal: Clin Chem
DOI: 10.1093/clinchem/hvaa107
sha: 
doc_id: 322641
cord_uid: mz0b91xr

nan

Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), the causative agent of coronavirus disease 2019 , has led to significant morbidity and mortality throughout the world (1) . In response to a lack of COVID-19 testing the FDA issued guidance regarding serologic assays, stating that although manufacturers could use the Emergency Use Authorization (EUA) pathway for approval, serologic assays could also be marketed in the US bypassing this approval process (2) . This has led to a dramatic availability of serologic tests, most of which from companies with little to no track record in the in vitro diagnostic market. At the time of writing, more than 100 manufacturers have notified the FDA that they are offering / plan to offer serologic testing in the US and only 4 have received EUA clearance. (3) . This represents an unprecedented release of manufacturer developed laboratory tests that have not been reviewed by the FDA. Some have appeared in non-peer reviewed preprint servers and have both gained attention and generated criticism in the lay press. Nonetheless, calls for serologic testing for COVID-19 have continued to gain momentum. Serology has been suggested to play three roles in the COVID-19 pandemic; 1) diagnosis, 2) identification of convalescent plasma donors, 3) screening populations with the purpose of determining exposure and immunity.

Diagnosis A common question from physicians in our hospital network is if serology can be used for diagnosis. Serologic assays are unlikely to be useful for this application.

Early reports indicate patients become positive for anti-SARS-CoV-2 IgG approximately 7-14 days after onset of symptoms (4) . In our hospital, the median time from onset of symptoms to patient presentation is 3 days (mean ~5 days), making serologic diagnosis unlikely at the time of initial patient evaluation. Many have suggested the use of anti-SARS-CoV-2 IgM to detect early infections and many lateral flow-based devices include IgM. However, most IgM based assays suffer from higher false positive rates relative to IgG based assays (5) . Due to these limitations molecular techniques are favored for the diagnosis of COVID-19. However, sensitivity limitations have been noted with molecular testing, directing providers to alternate means of testing in PCR-negative patients with compatible symptoms. The utility of serologic testing in this setting has yet to be established. In our experience, testing of patients previously negative by COVID-19 PCR revealed a small subset of serologically positive patients (~1% of symptomatic, PCR negative patients). While the exact disease status of these patients is difficult to adjudicate, these findings suggest a limited niche for diagnostic serologic testing. This Presumably, patients with high anti-SARS-CoV-2 titers would be the ideal donor population, making this a useful application. Further studies are needed to demonstrate if antibodies against SARS-CoV-2 are neutralizing and provide protection at the same level at which they are detected. Furthermore, since many serologic assays do not provide quantitative results, there are limited mechanisms for distinguishing between donors with high titers of antibody versus those with low titers. If there is an intent to convert qualitative assays to quantitative assays for this purpose they must be evaluated as such (including an assessment of precision spanning the reportable range).

Population Screening: Population screening has been proposed to identify exposed patients that are presumptively immune to SARS-CoV-2 secondary to exposure and to quantify the prevalence of exposure within a population for epidemiologic purposes.

While specific guidelines regarding how "presumptive immunity" will be determined and used do not exist, this potential use has generated the most excitement in the lay public.

Members of the White House Coronavirus Task Force have touted serologic assays to allow protected Americans to return to the workplace and rekindle the economy (7).

Unfortunately, it is not known if anti-SARS-CoV-2 antibodies are protective at this time, although the general assumption is that the presence of antibodies will provide at least some immunity. Though this is only an assumption, it is not without merit given what is known about other respiratory viruses (i.e. influenza) as well as the evidence the convalescent plasma may be an effective therapy (8) .

The challenge with this approach is that even in areas with reported extensive spread of COVID-19, seroprevalence is likely low. Even a laboratory test with excellent sensitivity and specificity is often unhelpful in low-prevalence settings. Here specificity and the incidence of false positive results must be accurately determined. If the prevalence of COVID-19 in the population is 20% a test with a sensitivity and specificity of 98% will make the value of a positive result (PPV) 92.5% (Figure 1 ). In New York City, the estimated prevalence is 1.69% (9), therefore if a serologic test has a specificity of 99.5%, the PPV is 76.6%. However, if the specificity is lowered to 98%, the PPV plummets to 45%; or ~1:2 positive results is a false positive. In such a scenario one half of those cleared to return to work would not have SARS-CoV-2 specific antibodies and would be at risk of acquiring and transmitting infection. In cities and states with lower prevalence such as Missouri (0.1%) (10), even a specificity of 99.5% will result in a PPV of 16.4%. , Understanding the correlations between prevalence, specificity, and PPV is a necessity before implementing serologic testing for screening.

The importance of specificity of serologic tests for screening low prevalence populations was recently demonstrated in a non-peer reviewed publication (11) . 3,300 residents in California were screened for antibodies to SARS-CoV-2 using a device without an EUA.

The authors found that 1.5% of those screened were positive for SARS-CoV-2 antibodies and, after analysis, found the estimated prevalence to be 2.4%. Using 30 specimens from patients pre-COVID-19 in combination with the manufacturers claims, the authors asserted a sensitivity of 80.3% and specificity of 99.5%. However, the 95% confidence interval of the specificity was 98.2-99.9. The authors acknowledged that the device had not been adequately validated and that if the specificity was near98%, the estimated prevalence would fall from 2.4% to <1%, with a 95% CI reaching 0%.

What could be the reason for these false positive results? Given the homology of SARS-CoV-2 to other coronaviruses, it is likely that antigens used as targets in poorly designed assays will cross react. This risk is exaggerated in older populations who are likely to have been exposed to a wider variety of coronaviruses (12) . Many serologic assays also cross-react in patients with EBV, rheumatoid factor, and heterophile antibodies. Careful antigen selection in COVID-19 serological assays is required to avoid cross reactivity of anti-seasonal coronavirus antibodies. If validation studies are not designed appropriately (ie. if only young, asymptomatic pre-pandemic patients are used as the negative population), then these limitations may not be thoroughly vetted and specificity of the assay may be grossly overestimated.

Given the lack of scientific rigor regarding the assessment of available serologic assays, it is a necessity that the laboratory community provide thorough examination of these methods and clearly articulate to providers their utility. This is particularly true of assays without an EUA whose performance may be variable. The Clinical and Laboratory Standards Institute (CLSI) recommends that sensitivity be assessed by analyzing at least 50 positive specimens from patients with confirmed disease (13) . Ideally this would be performed serially to determine the sensitivity at various days post-symptom onset. The CLSI also recommends 50 specimens to determine clinical specificity. While these recommendations are helpful, the rigor of the validation should ultimately be dependent on the intended test population (13) . For population-based screening with low seroprevalences, the specificity needs to be >99% with small confidence intervals (ie.

between 99.0-99.9) to ensure a high PPV. This would require a negative control population exceeding 750 people (14) or 2000 people (15) In conclusion, while serologic assays have generated much hype, there is a need for data to support their clinical utility. When not properly evaluated they have the potential to misdiagnose and misinform. It is also crucial for laboratories to rigorously validate assays to assure they are suitable for their ultimate use. . Furthermore, as the general public becomes enamored with the promise of COVID-19 serologic testing, it is the responsibility of laboratory professionals to remind everyone of the peril. 

Covid-19 -Navigating the Uncharted

Policy for Diagnostic Tests for Coronavirus Disease-2019 during the Public Health Emergency -Immediately in Effect Guidance for Clinical Laboratories, Commercial Manufacturers, and Food and Drug Administration Staff

US Food & Drug Administration. Emergency Use Authorizations

Severe Acute Respiratory Syndrome Coronavirus 2-Specific Antibody Responses in Coronavirus Disease

Immunoglobulin M for Acute Infection: True or False?

Infectious Diseases Society of America

Remarks by President Trump, Vice President Pence, and Members of the Coronavirus Task Force in Press Briefing

Treatment of 5 Critically Ill Patients With COVID-19 With Convalescent Plasma

COVID-19: Data

COVID-19 Outbreak

A serological assay to detect SARS-CoV-2 seroconversion in humans. medRxiv

Prevalence of antibodies to four human coronaviruses is lower in nasal secretions than in serum

User Protocol for Evaluation of Qualitative Test Performance. CLSI [Internet

Assessment of the Diagnostic Accuracy of Laboratory Tests Using Receiver Operating Characteristic Curves. CLSI [Internet

The Use of Confidence of Fiducial Limits Illustrated in the Case of the Binomial