key: cord-0877465-w9belk9j authors: Strand, Rasmus; Thelaus, Louise; Fernström, Nils; Sunnerhagen, Torgny; Lindroth, Ylva; Linder, Adam; Rasmussen, Magnus title: Rapid diagnostic testing for SARS‐CoV‐2: Validation and comparison of three point‐of‐care antibody tests date: 2021-03-05 journal: J Med Virol DOI: 10.1002/jmv.26913 sha: 91afffcd8e067b877e5b23101b8238c65d2018dc doc_id: 877465 cord_uid: w9belk9j With the emergence of severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2), a need for diagnostic tests has surfaced. Point‐of‐care (POC) antibody tests can detect immunoglobulin (Ig) G and M against SARS‐CoV‐2 in serum, plasma or whole blood and give results within 15 minutes. Validation of the performance of such tests is needed if they are to be used in clinical practice. In this study we evaluated three POC antibody tests. Convalescent serum samples from 47 reverse transcription polymerase chain reaction (RT‐PCR) verified coronavirus disease 2019 (COVID‐19) patients collected at least 28 days post RT‐PCR diagnosis as well as 50 negative pre‐COVID‐19 controls were tested. The three tests (denoted the J‐, N‐ and Z‐tests) displayed the sensitivities of 87, 96 and 85 percent, respectively, for the detection of IgG. All tests had the same specificity for IgG (98 percent). The tests did not differ significantly for the detection of IgG. The sensitivities for IgM were lower (15, 67 and 70 percent) and the specificities were 90, 98 and 90 percent, respectively. The positive and negative predictive values were similar among the tests. Our results indicate that these POC antibody tests might be accurate enough to use in routine clinical practice. This article is protected by copyright. All rights reserved. Since its discovery in early 2020, the coronavirus disease 2019 (COVID-19) pandemic caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has swept across the world in an unprecedented fashion and created a massive need for rapid and accurate diagnostic tests. The currently recommended way to diagnose active infection is via reverse transcription polymerase chain reaction (RT-PCR) based methods. By detecting viral RNA, RT-PCR has become the reference method to which other methods are compared. Despite this, results from some studies report varying sensitivity for RT-PCR, especially if performed on a patient in the incubation period of the disease (1) . An alternative way of identifying individuals who have been infected is by analyzing blood or serum for presence of SARS-CoV-2 specific antibodies. This can be accomplished through the use of for example point-of-care (POC) tests which are often of the lateral flow immunoassay type and work by detecting IgG and IgM against SARS-CoV-2 in serum, plasma or whole blood. Results are often available as soon as 10 minutes after initiation of testing. Prior studies have shown that most patients have developed antibodies two weeks after symptom onset (2, 3) . While these types of rapid diagnostic tests (RDT) may not be used to identify patients with an active infection, they can potentially be used to confirm whether or not the patient has undergone infection and developed antibodies. This can be especially valuable for finding infected individuals This article is protected by copyright. All rights reserved. who did not get tested with RT-PCR during the acute phase of infection due to exhibiting few or no symptoms. A rapid influx of POC antibody tests has hit the market. The reported sensitivity of these tests varies significantly, ranging from 39-100 percent, but the specificity is generally high, ranging from 90-99 percent (4) (5) (6) . In this study, we have compared and validated three different POC antibody tests. The study was conducted at the Department for Infectious Diseases, Skåne University Hospital, Lund, Sweden. Convalescent blood samples from patients with RT-PCR verified COVID-19 (n=47) were collected at least 28 days after the RT-PCR verified COVID-19 diagnosis. 23 (49%) of the patients were hospitalized, of whom 11 required treatment with oxygen. The blood was allowed to coagulate for one hour and centrifuged at 570xg for 10 minutes. Serum was frozen at -80°C until analysis. RT-PCR for SARS-CoV-2 was performed on nasopharyngeal swab samples with a modified in-house method in line with World Health Organization (WHO) guidelines as described by Corman et al. (7) . In brief, primer design and assay sequence are identical to the referred method. Our modifications constitutes changes in the thermal cycling for the E-and RdRP genes, utilizing 48°C for 10 minutes followed by 95°C for 10 minutes. We used an annealing temperature for the E gene of 55°C, and we used an amplification phase of 45 seconds for both genes. Finally, for the RdRP probe we used a concentration of 0.2 µM. The negative control group (n=50) was comprised of serum samples obtained from patients four to six weeks after discharge from in-hospital treatment for respiratory tract infections. Serum was collected as above between 1997 and 2007. The samples had been kept frozen at -80°C since collection. The three tests evaluated in this study were the SARS-CoV-2 immunoglobulin (Ig) G/IgM Antibody test (Colloidal Gold) from Joinstar Biomedical Technology Co. Categorical data were expressed as numbers and differences between data were analyzed using the Chi-square test. Fisher's exact test was used when comparing the values of two tests to each other. The specificities and sensitivities were calculated and presented with 95% confidence intervals within parentheses. The confidence intervals were estimated with the Clopper-Pearson method. Statistical analysis was performed with, and graphs were created using, GraphPad Prism version 8.3.1. A p-value <0.05 was considered statistically significant. Ethical approval was granted by the Swedish national ethics committee (2020-01747). Blood samples from patients with verified COVID-19 were collected after informed signed consent was obtained. Serum samples from pre-COVID-19 patients had been stored for research purposes as part of clinical routine. The samples were anonymized during handling in the laboratory. The N-test displayed the highest sensitivity for detecting IgG (96 (85-99) percent)). The observed sensitivities for the J-and Z-tests were 87 (74-95) and 85 (72-94) percent, respectively. There was no statistically significant difference in the ability to detect IgG between the three tests (p=0.4). All three tests displayed the same specificity (98 percent). Sensitivity and specificity for each test to detect IgG can be seen in Figure 1A and Table 1 . As for IgM, the N-and Z-tests displayed similar sensitivities: 67 (52-80) and 70 (55-83) percent, respectively. The J-test exhibited a lower sensitivity: 15 (6-28) percent. The N-This article is protected by copyright. All rights reserved. and Z-tests both differed significantly when compared to the J-test (p<.001), but they did not differ when compared to each other (p=0.8). The specificities were found to be 90 (78-97) percent for the J-test, 98 (89-100) percent for the N-test and 90 (78-97) percent for the Z-test. The tests did not differ significantly in terms of specificity (p=0.2). The sensitivity and specificity for each test can be seen in Figure 1B and Table 1 . Upon comparing the effect on the positive predictive value (PPV) when the prevalence of positive patients ranged between 0 and 25 percent, there was no noteworthy difference found between the tests. The graphical illustration of this is presented in Figure 1C . In this study, we have evaluated and compared three POC antibody tests designed to detect specific IgG and IgM antibodies against SARS-CoV-2. Due to the high contagiousness of SARS-CoV-2 (8), being able to identify patients who have undergone COVID-19 infection can be valuable during the process of differential diagnosis and for studying routes of transmission. A plethora of antibody tests have been developed for this purpose, but their sensitivity and specificity need to be validated in order for them to be used in routine clinical practice. Of the three evaluated tests, we found no significant differences between the tests regarding the sensitivity to detect IgG antibodies. Furthermore, there was no noteworthy difference between the tests when looking at the effect of prevalence on the PPV. A similar design has been employed in other studies, and while the sensitivity for detecting antibodies in convalescent samples varies between brands, the sensitivity for detecting antibodies in samples taken in the acute phase of the infection is generally low (4, 6) . As such, RDT POC antibody tests might not be suitable for confirming active infection in the emergency care setting. However, a positive IgG result from a test with high specificity can still potentially be used to rule out active infection. When compared to the results on the sensitivity to detect IgG against SARS-CoV-2 in convalescent samples from previous studies (4) (5) (6) , all three of the evaluated tests in this study had a relatively high sensitivity. A potential reason for this could be that we have evaluated the Accepted Article uncertain true sensitivity and specificity (10). However, as RT-PCR has become an established gold standard for many authors, for the sake of consistency with the reported findings from similar studies (4-6), we chose to also use sensitivity and specificity for the observed results in this study. Another factor potentially affecting seroconversion is disease severity. It has been shown that a more severe course of COVID-19 correlates to higher levels of antibody production (11). Of the 47 included RT-PCR positive patients, 23 were admitted. 77% (n=36) of the included patients with COVID-19 in this study had a mild disease course. Of the six discordant samples mentioned above, five came from patients with mild symptoms. The observed differences in sensitivity was thus likely due to relatively low antibody levels in some individuals with a mild course of COVID-19. Since the sensitivity for detecting IgG ranged between 85 and 96 percent between the three tests, we wanted to examine how this would impact the PPV. As seen in Figure 1C , there was no relevant difference in PPV even when plotting against a low prevalence. As the hypothesized prevalence of COVID-19 increases, the difference in PPV decreases. At a realistic prevalence of for example 10 percent, all three of the evaluated tests have a similar, and for clinical practices usable PPV of 83-84 percent. In addition to IgG this study also evaluates the ability to detect IgM for the three tests. All serum samples were collected a minimum of 28 days after diagnosis via RT-PCR testing. However, we cannot rule out the possibility that patients may have been infected before RT-PCR diagnosis. As the initial IgM response decreases over time, this could explain the relatively low sensitivity for IgM found for the three tests in this study. To accurately measure the IgM response the study should have been designed differently to include patients closer to the RT-PCR verified diagnosis. This, however, would have hampered our main objective which was to evaluate the presence of antibodies in convalescent samples. Some of the tested RT-PCR positive serum samples used to evaluate the three tests have also been used for the same purpose to evaluate the Z-test in another study. The sensitivity for detecting IgM in samples obtained from patients within two weeks of symptom debut was in that study 63 percent (12). Variation in False-Negative Rate of Reverse Transcriptase Polymerase Chain Reaction-Based SARS Tests by Time Since Exposure Severe Acute Respiratory Syndrome Coronavirus 2-Specific Antibody Responses in Coronavirus Disease Patients Antibody responses to SARS-CoV-2 in patients with COVID-19 Evaluation of eleven rapid tests for detection of antibodies against SARS-CoV-2 Antibody Tests in Detecting SARS-CoV-2 Infection: A Meta-Analysis. Diagnostics (Basel) Diagnostic performance of seven rapid IgG/IgM antibody tests and the Euroimmun IgA/IgG ELISA in COVID-19 patients Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR High Contagiousness and Rapid Spread of Severe Acute Respiratory Syndrome Coronavirus 2 Sensitivity and specificity for immunoglobulin (Ig) G antibodies found in serum samples for RT-PCR positive patients and control patients. (B) Sensitivity and specificity for IgM antibodies found in serum samples for RT-PCR positive patients and control patients. (C) Effect of prevalence on positive predictive value (PPV) for the detection of IgG within the range of 0-25 percent for the three evaluated point-of-care antibody tests. J, N and Z correspond to the SARS-CoV-2 immunoglobulin (Ig) G/IgM Antibody test (Colloidal Gold) from Joinstar Biomedical Technology Co The gathered data supporting the findings of this study are available from the authors upon request.