key: cord-0844286-l863bx66 authors: Bastola, Mrigendra M.; Locatis, Craig; Fontelo, Paul title: Diagnostic Laboratory Tests for COVID-19 in US: Methodology and Performance date: 2020-07-20 journal: Res Sq DOI: 10.21203/rs.3.rs-43374/v1 sha: 44506a2f32ea24c30bc8441009f6e465938c6d4d doc_id: 844286 cord_uid: l863bx66 Background: COVID-19 is a global pandemic caused by a new coronavirus strain. Innovative tests have been developed to diagnose and characterize the spread of COVID-19. Only a few studies have reported the diagnostic value of currently available tests. The diagnostic performance of the tests is a major concern after the recent resurgence in COVID-19. Methods: Published papers and FDA data on the currently available tests were used for analysis. Likelihood ratios, and predictive values of tests were computed. Only FDA approved tests were included. RT-PCR performance among different specimen types were also explored. Main results: All the published reports on the COVID-19 tests reported RT-PCR as the validation tool for their results. Not all available COVID-19 tests reported their sensitivity and specificity. Among the publications which reported, the positive likelihood ratio ranged between 0.15 to 0.88 and tests had high negative likelihood ratio (0.99). Conclusion: Although most recent publications showed high positive and negative likelihood ratios and high predictive values, the publications on test accuracy and validity have limited scope primarily due to their small sample size and insufficiencies in methodology and published data. Although most lab tests reported high sensitivity and specificity, false omission and false discovery rates were found notable in several COVID-19 lab tests. These results suggest need for caution on test results’ interpretation. Practitioners also need to integrate evidence that is evolving rapidly. The virus SARS-CoV-2, which causes the illness COVID-19 was rst detected in late 2019. To date (July 1, 2020), more than 10 million COVID-19 cases have been reported with more than half a million deaths worldwide, and the numbers are expected to increase [1] . The rapid detection of COVID-19 cases in the United States will assist in case diagnosis, contact tracing, and help mitigate further spread. A public health emergency determination was made on February 4, 2020 authorizing of emergency use of invitro diagnostic tests for the diagnosis of COVID-19 [2] . As allowed by the FDA, 7 states in USA permit certi ed laboratories to develop and perform COVID-19 tests without ling an Emergency Use Authorization (EUA) [3] . When validated, the FDA allows the development and distribution by commercial manufacturers, or development and use by laboratories of serology tests to identify antibodies to SARS-CoV-2 [3] . As of May 29, 2020, there were 63 test kit manufacturers and commercial laboratories offering COVID-19 testing listed on the FDA website [2] . 51 were molecular tests and 12 serology tests. A few companies had developed both. All tests that were listed were certi ed for use only in CLIA (Clinical Laboratory Improvement Amendments) to perform high complexity tests, 21 in the moderately complex test category and only 4 were deemed eligible for use in patient care settings [1] . Several of these laboratory tests are currently allowed under EUA, and state governments can perform tests in certi ed laboratories and physician provider o ces. The test reports from these non-EUA tests must indicate that the test has not been FDA reviewed, a negative test does not rule out COVID-19, the test report must be used with clinical features for diagnosis of COVID-19 and the test reports might be due to present or past infection with SARS-CoV-2 coronavirus strains [3] . Although this information cautions users, it provides little help in understanding the performance of the available COVID-19 tests. The performance of these available tests is a major concern after the recent resurgence in COVID-19 cases. This brief report of published articles and FDA data explores the methodology, accuracy, sensitivity, and likelihood ratios and predictive values of positive and negative tests of these FDA approved tests and EUA tests. The review provides insights into false negatives, low sensitivities, predictive values, methodology and questionable performance of currently most used COVID-19 tests and specimens, which are relevant in the context of the rising pandemic. A PubMed search for journal publications on the topic, "laboratory diagnosis of COVID-19" completed on April 19, 2020, returned 319 articles. This report only included journal articles that addressed laboratory diagnosis and excluded publications that discussed diagnosis based on clinical features and imaging studies. Only laboratory tests were reviewed and included most used and FDA recognized molecular-bases tests, serological tests, ELISA based tests and chemiluminescence tests. The positive and negative likelihood ratios, and positive and negative predictive values of tests were calculated based on the current national incidence of 7542.61/ per million (national average as of July 1, 2020, see also Table 1& 2). The prevalence needed in the calculation process was replaced by the incidence rates because the COVID-19 has same prevalence and incidence rates for 2020, and there were no prior cases in US before January 2020 [1] . All tests were either FDA approved, or currently in the approval process. Tests that had not provided their data on sensitivity and speci city were excluded in this review (As of July 1, 2020). Additionally, tests approved for research surveillance purposes only were excluded in this study. Because most of the publications reported only sensitivity and speci city and did not report either true or false data on positive and negative tests, this report calculated predictive values directly from sensitivity and speci city values. Final selection included seven papers and online publications, which su ciently reported relevant data on test performance and test methodology (Table 1& 2) . Included studies, data collected, and results are shown in Table 1 & 2. All the tests used RT-PCR for validation. Only two studies directly compared the accuracy rates for nasopharyngeal versus oropharyngeal swabs, and both had insu ciencies in sample size, de ciencies in reporting false/true values for positive and negative tests, and description of methods [4] . The sensitivity of the RDTs varied between 87 to 97% while the speci city varied between 96 to 100%. Among the publications which reported, the positive likelihood ratio ranged between 0.15 to 0.88 and tests had high negative likelihood ratio (0.99). Several tests had high false detection rate, and some had high false omission rates (Table 1) . Of the samples collected for RT-PCR, bronchoalveolar lavage had the highest sensitivity (93%) and speci city (100%), while nasopharyngeal and sputum had higher sensitivities compared to oropharyngeal swabs ( Table 2 ). Although sensitivity and speci city of the tests have not been well established, extensive testing has been advocated for early diagnosis and quarantine of the COVID-19 patients to help mitigate the spread of the disease, as practiced in South Korea [5] . Nevertheless, the published data on sensitivity and speci city of the commonly used tests are neither consistent nor clear, with not enough published data on false positives and false negatives [6] . Apart from lymphocytopenia (in 83% of COVID-19 positives), the rest of the routine blood work was inconsistent and nonspeci c [7] . Currently, vaccines are still undergoing clinical trials [8] . Levels of neutralizing antibodies needed, and protective herd immunity remains uncertain in COVID-19. Published reports suggest that herd immunity may develop only after extensive vaccination or after at least 68% of the population recover from COVID-19 [9] . Although the FDA has recently approved home based test kits for COVID-19, their performance is still unknown [10, 11, 12] . This brief report summarizes published results of currently approved COVID-19 tests has several implications. The positive and negative likelihood ratios mirror the sensitivities and speci cities because most of the studies exclusively reported only true positive and true negative results (Table 1&2 ) [13] . A caveat with molecular testing is that the viral RNA might be undetectable within a week of infection and antibodies IgG and IgM rise after day 9 onwards [14] . Tests which have high sensitivity are best used for screening symptomatic patients, while tests with highest speci city are best for con rming diagnosis, so patients with positive tests can treated or quarantined [15] . Both false negative and false positive tests may lead to erroneous decisions in managing patients, such as contact tracing, isolation, and treatment [6, 16] . Additionally, false negatives may have greater importance among health care workers who might otherwise appear to be asymptomatic. Moreover, nding high SARS-CoV-2 RNA in feces that was not previously reported, has raised public health concerns [9] . There is limited published data on tests that are approved outside the US. Comparable results are seen on the results of this study to these limited data on tests approved outside USA reported 87 to 100 percent sensitivity and reported range from 98.7 to 100% speci city [11] . Tests having high sensitivity providing quick results in physicians' o ces or in clinic settings can be valuable additional tools in tracking COVID-19 patients and asymptomatic individuals and can help provide an epidemiologic pro le of a community. Since no single test can attain 100% sensitivity and 100% speci city, the best approach to testing in COVID-19 would probably come from a combination of tests with high sensitivity in the rst and high speci city in the second [17] . Cross-reactivity with other types of non-virulent coronavirus has not been established for RDTs, hence the caution is needed in reporting results. The value of the antibody-based tests on diagnosing individuals with COVID-19 in recovery or past infection with COVID-19 is still not fully understood. Furthermore, longterm immunity after COVID-19 infection is still uncertain, which could undermine the effectiveness of a future vaccine if no long-term or strain-speci c neutralizing antibodies are produced in adequate quantities. Published results showed that several sample types for RT-PCR based tests had low sensitivity and signi cant false omission and false discovery rates (Table 1 and Table 2 ). These have been explained by specimens collected too early when viremia was low or after patients presented clinically with signs and symptoms and may have passed the active viremia phase, or due to technical issues. Only a few studies support the use of saliva and nasopharyngeal samples for testing. Sputum specimens and nasopharyngeal swabs yielded 69% and 63% sensitivity, respectively, suggesting that sputum specimens might have adequate viral nucleic acids for detection using the RT-PCR assay [18] . Moreover, salivary viral load was highest during the rst week after symptom onset but subsequently declined over time [18] . In a large study conducted in China, the positive rate was 57% for selected individuals presenting with fever in clinics at the epicenter of the epidemic [19] . Older age was correlated with higher viral load [19] . In another study, the presence of antibodies and seropositivity rates were higher for anti-RBD (receptor binding domain) IgG and anti-NP (nucleoprotein) IgG, than anti-RBD IgM and anti-NP IgM [20] . Viral RNA has been reported 37 days after symptom onset [21] . Whether patients are still infective at this point is unknown. The results of the ongoing pandemic in this review indicate that the best sensitivity is attained using bronchoalveolar uid (93%) followed by saliva (92%), sputum (69%) then nasopharyngeal swabs (63%) and serum (41%) ( Table 2 ) . Feces only yielded 29% positivity. Perplexing is the low sensitivity of blood samples (1%). Specimen collection and processing and specimen preservation and transport could have factored signi cantly into the results found. The ndings of this review paper support the emphasis on COVID-19 testing made by WHO, however, even the level of diagnostic accuracy might not be su cient for effective control of the pandemic [22] . A recent study which analyzed samples from known COVID-19 individuals, percent seropositive increased with time interval, peaking at 81.8-100.0% in samples taken >20 days after symptom onset [23] . A few laboratories were developing molecularbased tests that were authorized for use in their laboratory only. These are laboratories that perform high complexity tests authorized through EUAs. Some tests were developed outside the US and have been purchased by some states or imported by companies in the US. These tests may not have been authorized by the FDA or are in the process of applying for EUAs. As testing methods are continuously evolving, newer tests may attain higher speci cities and sensitivities, either alone or in combination. Limited published data on performance of the lab tests for COVID-19, including data on true and false positive rates on positive and negative tests is a major limitation of the study, which led authors to rely on direct computation of predictive values from reported sensitivity and speci city values. The results in this review must be interpreted in the context of exposure history and other supporting evidence. Multiple factors besides test performance and false omission rates discussed in review are possibly aiding undetected COVID-19 cases. Although not necessarily a limitation in this review, geographic locations could have introduced their own unique demographic characteristics in the studies. As in most laboratory tests, the predictive value, both positive and negative, depend greatly on the prevalence of disease in the population. This study also assumes that the sample collection and processing methods were comparable in all included studies. This review included only a select number of articles, and could not have included all publications, which are added daily and at a rapid pace. This study used the most recent published prevalence rate from CDC in the USA, which is subject to change if the pandemic progresses, and higher prevalence gives higher performance for the COVID-19 tests. In Tests approved in countries other than US were excluded in this study due to limited published data on their performance. Most of the recently published studies of tests on COVID-19 are limited in applicability primarily because of their small sample size, insu cient methods, and available data in publications. Although most lab tests reported high sensitivity and speci city, false omission and false discovery rates were found notable in several COVID-19 lab tests. However, there are suggestions of which specimen sources might yield the best sensitivity and speci city. The results must be interpreted with caution and practitioners need to integrate the most recent evidence that is evolving rapidly. A careful selection of tests with high sensitivity and speci city would be highly bene cial in specimen selection and in efforts to mitigate the spread of COVID-19. Availability of data and materials: The data used in this review are extracted from published research. Additionally, the datasets used, and data analysis output are available from the corresponding author upon reasonable request. Competing interests: Authors declare no competing interests. Funding: This research was supported by the Intramural Research Program of the National Institutes of Health (NIH), National Library of Medicine (NLM), and Lister Hill National Center for Biomedical Communications (LHNCBC). The funding body did not play any role in the in the design of the study, collection, analysis, and interpretation of data, or in writing the manuscript. Author's contributions: MB collected and analyzed the data, assisted in the study design and in drafting the manuscript. CL assisted in the data analysis, and in drafting the manuscript. PF conceived of the study and study design, participated in the data analysis, and drafted the manuscript. All authors read and approved the nal manuscript. ArcGIS Dashboards Emergency Use Authorizations Food and Drug Administration FAQs on Diagnostic Testing for SARS-CoV-2 Comparative accuracy of oropharyngeal and nasopharyngeal swabs for diagnosis of COVID-19 -CEBM Dealing with False Negative COVID-19 Test Results False-negative COVID-19 test results may lead to false sense of security Viral dynamics in mild and severe cases of COVID-19. The Lancet Infectious Diseases Herd immunity -estimating the level required to halt the COVID-19 epidemics in affected countries Detection of SARS-CoV-2 in Different Types of Clinical Specimens Global Progress on COVID-19 Serology-Based Testing COVID-19) Update: Serological Test Validation and Education Efforts Food and Drug Administration Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR Serological and molecular ndings during SARS-CoV-2 infection: the rst case study in Finland Guidelines for Laboratory Diagnosis of Coronavirus Disease 2019 (COVID-19) in Korea Potential false-negative nucleic acid testing results for Severe Acute Respiratory Syndrome Coronavirus 2 from thermal inactivation of samples with low viral loads The Laboratory Diagnosis of COVID-19 Infection: Current Issues and Challenges Comparison of throat swabs and sputum specimens for viral nucleic acid detection in 52 cases of novel coronavirus (SARS-Cov-2)-infected pneumonia (COVID-19) Positive rate of RT-PCR detection of SARS-CoV-2 infection in 4880 cases from one hospital in Temporal pro les of viral load in posterior oropharyngeal saliva samples and serum antibody responses during infection by SARS-CoV-2: an observational cohort study. The Lancet Infectious Diseases Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. The Lancet PCR Assays Turned Positive in 25 Discharged COVID-19 Patients WHO Director-General's opening remarks at the media brie ng on COVID-19 -16 Test performance evaluation of SARS-CoV-2 serological assays Geographic Differences in COVID-19 Cases, Deaths, and Incidence -United States Correlation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases Consistent detection of 2019 novel coronavirus in saliva Viral load dynamics and disease severity in patients infected with SARS-CoV-2 in Zhejiang province, China Food and Drug Administration Acknowledgment: Not applicable.Disclaimer: The views and opinions of the authors expressed herein do not necessarily state or re ect those of the National Library of Medicine, National Institutes of Health, or the US Department of Health and Human Services.