key: cord-339711-f7xifne8 authors: Bal, A.; Pozzetto, B.; Trabaud, M.-A.; Escuret, V.; Rabilloud, M.; Langlois-jacques, C.; Paul, A.; Guibert, N.; D'aubarde, C.; Massardier, A.; Boibieux, A.; Morfin, F.; Pitiot, V.; Gueyffier, F.; Lina, B.; Fassier, J. B.; Assant, S. title: Evaluation of high-throughput SARS-CoV-2 serological assays in a longitudinal cohort of mild COVID-19 patients: sensitivity, specificity and association with virus neutralization test date: 2020-09-30 journal: nan DOI: 10.1101/2020.09.30.20194290 sha: doc_id: 339711 cord_uid: f7xifne8 Objectives: We evaluated widely-used SARS-CoV-2 serological tests and their potential association with virus neutralization test (VNT) in a cohort of mild COVID-19 patients. Methods: A total of 439 specimens were longitudinally collected from 76 healthcare workers with RT-PCR-confirmed mild COVID-19. Nine serological assays developed by leading global companies (Abbott, DiaSorin, Siemens, Bio-Rad, Wantai, bioMerieux, Euroimmun) were assessed. For each test the sensitivity to detect SARS-CoV-2 antibodies was determined weekly after symptom onset. Correlation and concordance were assessed using the Spearman and Cohen Kappa coefficients, respectively. Positive percent agreement and negative percent agreement (NPA) with the VNT were also determined. Results: The Wantai Total Ab assay targeting the receptor binding domain (RBD) within the S protein presented the best sensitivity at different times during the course of disease. The best correlation between antibody level and neutralizing antibody titer was found with the Euroimmun S1-based IgA assay (Spearman coefficient [95%CI]: 0.71 [0.61-0.79]). A moderate concordance (Kappa [95%CI]: 0.43[0.23-0.63]) as well as the lowest NPA (33%) was found between the Wantai Total Ab assay and the VNT. Compared to the Wantai Total Ab assay, other total Ab or IgG assays targeting the S or the RBD (bioMerieux, DiaSorin, Siemens,) were more concordant with the VNT (Kappa>0.7 for the three tests) and had a higher NPA (range: 90% to 97%). Conclusions: Although some assays presented a better concordance with VNT than others, the present findings emphasize that commercialized serological tests including those targeting the RBD cannot substitute VNT for the assessment of functional antibody response. The evaluation of the humoral immune response to SARS-CoV-2 with serological tests is 59 crucial to further manage the coronavirus disease 2019 (COVID-19) pandemic. Serological 60 testing represents an easy to implement and cost-effective method allowing to rapidly identify 61 individuals exposed to the virus [1, 2] . Over the last few months, a large number of SARS-62 CoV-2 commercial assays have been evaluated for their ability to detect specific antibodies 63 [3] [4] [5] [6] [7] [8] . However, the detection of specific SARS-CoV-2 antibodies does not indicate whether 64 or not the antibodies are functional for neutralizing the virus. In association with the 65 evaluation of other immune responses, such as cellular immunity, the determination of 66 neutralizing antibody titer is important to evaluate the protective immunity to SARS-CoV-2 67 after infection and therefore the risk of reinfection [9] [10] [11] . While the comparison of sensitivity 68 and specificity of serological tests has been increasingly studied, the association between the 69 results obtained with commercial tests and the virus neutralization test (VNT) has been 70 explored in only a few studies, and mostly among severe COVID-19 patients [12, 13] . VNT is 71 considered as the reference to assess the functional ability of antibodies to block the entry of 72 the virus into human cells [14] . However, such an assay requires living virus manipulated in a 73 biosafety level 3 (BSL3) facility that needs trained staff and specific equipment, and which is 74 a tedious and time-consuming method. The first study exploring the association of 75 commercial serological assays and VNT claimed that the Wantai Total Ab assay detecting 76 total antibodies directed against the SARS-CoV-2 receptor binding domain (RBD) had the 77 best characteristics to detect functional antibodies at different stages and severity of disease 78 [12] . The RBD, within the sub-unit S1 of the spike protein, enables the viral entry into human 79 cells by fixing to the angiotensin-converting enzyme 2 (ACE2) receptor [15] . As emphasized 80 by the authors [12] , there is an urgent need for further studies addressing the performance of 81 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted September 30, 2020. . https://doi.org/10.1101/2020.09.30.20194290 doi: medRxiv preprint A total of 9 serological assays developed by leading global companies in the field (Abbott, 104 DiaSorin, Siemens, Bio-Rad, Wantai, bioMérieux, Euroimmun) were investigated according 105 to the protocol recommended by each manufacturer (characteristics are summarized in Table 106 1). Positivity was established according to threshold value recommended by each 107 manufacturer. As previously suggested, we also evaluated a cut-off (OD ratio ≥ 10) to 108 indicate the presence of protective antibodies for the Wantai Total Ab assay [12] . 109 The VNT used for the detection and titration of neutralizing antibodies was performed as 110 previously described [17] . Briefly, a ten-fold dilution of each serum specimen in culture 111 medium (Dulbecco's Modified Eagle Medium containing antibiotics and 2% foetal calf 112 serum) was first heated for 30 min at 56°C to avoid complement-linked reduction of the viral 113 activity. Serial two-fold dilutions (tested in duplicate) of the serum specimens in culture 114 medium were mixed at equal volume with the live SARS-CoV2 virus. After gentle shaking 115 and a contact of 30 minutes at room temperature in plastic microplates, 150 µL of the mix was 116 transferred into 96-well microplates covered with Vero E6 cells. The plates were incubated at 117 37°C in a 5% CO 2 atmosphere. The reading was evaluated microscopically 5 to 6 days later 118 when the cytopathic effect of the virus control reached 100 TCID 50 /150 µL. Neutralization 119 was recorded if more than 50% of the cells present in the well were preserved. The 120 neutralizing titer was expressed as the inverse of the higher serum dilution that exhibited 121 neutralizing activity; a threshold of 20 was used. All experiments were performed in a BSL3 122 laboratory. The comparison of this VNT with a standardized assay using retroviruses pseudo-123 typed with the SARS-CoV-2 S viral surface protein found a high correlation and concordance 124 [17] . 125 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted September 30, 2020. Total Ab assay, the differences were significant before 14 days post-symptom onset with all 151 other assays, except with the Euroimmun and Bio-Rad assays; after 14 days post-symptom 152 onset, the differences were significant with all other assays. 153 Regarding specificity, no false positive result was found using 30 pre-pandemic sera, although 154 3 samples gave a borderline ratio (between 0.8 and 1.1) with the Euroimmun IgA assay 155 (supplementary table 1) . 156 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted September 30, 2020. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The best correlation between commercial kits and VNA was found with the Euroimmun S1- Wantai Total Ab; 18 these had an OD ratio < 10. Of note, 9/30 samples were collected more 182 than 14 days post symptom onset. Regarding the samples with an OD ratio < 10 with the 183 Wantai Total ab assay (39/140, 28%), all had a low level of neutralizing antibodies (range: 184 20-50; Figure 1 ). 185 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this this version posted September 30, 2020. . The NPA ranged from 33.3% [21.1-48 .3] for the Wantai Total Ab assay to 96.8% for the 186 DiaSorin and was < 90% for 7/9 assays. The PPA was > 90% for all tests except the DiaSorin 187 and the two IgM based assays (Wantai and bioMérieux) ( Table 2) . 188 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this this version posted September 30, 2020. . These findings are highly consistent with those of the present study that found a NPA below 207 90% for all tests except for bioMérieux IgG and DiaSorin. In contrast, it is interesting to note 208 that the Wantai Total Ab had the lowest NPA (33%). Furthermore, the concordance between 209 VNT and the Wantai Total Ab assay was only moderate while the concordance was 210 substantial with bioMérieux IgG, DiaSorin, Siemens, Abbott, Euroimmun and Bio-Rad. The 211 low NPA and moderate concordance noticed for the Wantai Total Ab might be partially 212 explained by the excellent ability of this test to detect RBD-specific antibodies at the very 213 early phase of infection, irrespective of their neutralizing properties in line with the delay 214 required for antibody maturation [18] . In the first study comparing VNT with commercialized 215 tests, the authors found that the Wantai Total Ab assay had the best characteristics to detect 216 functional antibodies in different stages and severity of disease [12] . However the median 217 interval between the onset of symptoms and sample collection was 43 days for mild patients 218 (n=71 samples) and thus the antibodies could be detected with both the Wantai Total Ab and 219 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted September 30, 2020. . the VNT assay at this time. As the authors used VNT as the gold-standard for sensitivity 220 assessment, this explains the difference in findings with the present study [12] . 221 Importantly, as previously reported by others [18, 19] , not all RBD-binding antibodies have 222 neutralizing properties which is consistent with that reported herein regarding the RBD-based 223 assays that do not have perfect concordance with VNT. Conversely, serological assays 224 targeting a region other than the S protein may be associated with functional information, as 225 previously reported [14, [20] [21] [22] . In the present study, the Abbott and Bio-Rad assays directed 226 against the N protein presented a substantial concordance with VNT as N-directed and RBD-227 neutralizing antibodies can be produced concomitantly over the course of the disease. 228 In addition to the different targeted antigens, the heterogeneity in assay performance found 229 herein could also be explained by various factors including the detected isotypes. The present study does, however, have certain limitations. For instance, specificity was not 236 been extensively studied; yet the Euroimmun IgA assay seemed to have the worst specificity, 237 which is consistent with previous studies reporting a lack of specificity for this assay [5, 6, 12] . The results presented herein obtained from mild COVID-19 patients confirm that, for 245 exposure assessment, the Wantai Total Ab assay should be preferred to other commercial kits 246 due to a very high sensitivity. For evaluating protective immunity, the Wantai Total Ab assay 247 with an optimized cut-off or other tests targeting the S protein as Euroimmun, DiaSorin or 248 bioMérieux IgG could be more useful, notably to screen serum specimens candidate for the 249 presence of neutralizing antibodies. However, these tests or others cannot substitute a VNT 250 for assessing functional antibody response; neutralizing assays remain the gold standard and 251 easy-to-use tests, such as those based on pseudoviruses [6,17,24], should be developed and 252 standardized. Furthermore, the recent development of surrogate virus neutralization tests 253 based on antibody-mediated blockage of the interaction between ACE-2 receptor and the 254 RBD is very promising as they were designed in an ELISA format enabling high-throughput 255 testing [19, 25] . 256 In conclusion, the present study provides original data concerning the performance of widely-257 used serological tests, which could help diagnostic laboratories in the choice of a particular 258 assay according to the intended use. 259 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted September 30, 2020. . Antonin Bal has received grant from bioMérieux and has served as consultant for bioMérieux 299 for work and research not related to this manuscript. Sophie Trouillet-Assant has received 300 research grant from bioMérieux concerning previous works not related to this manuscript. 301 The other authors have no relevant affiliations or financial involvement with any organization 302 or entity with a financial interest in or financial conflict with the subject matter or materials 303 discussed in the manuscript. 304 Funding 305 This research is being supported by Hospices Civils de Lyon and by Fondation des Hospices 306 Civils de Lyon. 307 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted September 30, 2020. . Serology assays to manage COVID-19 The important role of serology for COVID-19 control Comparison of eight 313 commercial, high-throughput, automated or ELISA assays detecting SARS-CoV-2 IgG 314 or total antibody Antibody response against SARS-CoV-2 spike protein and 318 nucleoprotein evaluated by 4 automated immunoassays and 3 ELISAs 321 Diagnostic performance of seven rapid IgG/IgM antibody tests and the Euroimmun 322 IgA/IgG ELISA in COVID-19 patients Validation of a 325 commercially available SARS-CoV-2 serological immunoassay Performance Characteristics of Four High-328 Throughput Immunoassays for Detection of IgG Antibodies against SARS-CoV-2 331 Diagnostic accuracy of serological tests for covid-19: systematic review and meta-332 analysis Neutralizing antibodies correlate with protection from SARS-CoV-2 in humans during a 335 fishery vessel outbreak with high attack rate SARS-CoV-2 T cell immunity: Specificity, function, 338 durability, and role in protection A systematic review of antibody mediated immunity to coronaviruses: kinetics, 342 correlates of protection, and association with severity An evaluation of COVID-19 serological assays informs future diagnostics and 346 exposure assessment 349 Association between SARS-CoV-2 neutralizing antibodies and commercial serological 350 assays Neutralizing Antibodies against SARS-CoV-2 and Other 352 Human Coronaviruses A pneumonia outbreak 355 associated with a new coronavirus of probable bat origin Assessment of Serological Techniques for Screening Patients 358 Regarding COVID-19 (COVID-SER): a prospective multicentric study A longitudinal study 361 of SARS-CoV-2 infected patients shows high correlation between neutralizing 362 antibodies and COVID-19 severity Human neutralizing antibodies elicited 365 by SARS-CoV-2 infection A SARS-CoV-2 surrogate 368 virus neutralization test based on antibody-mediated blockage of ACE2-spike protein-369 protein interaction Coronaviruses pandemics: Can neutralizing antibodies 372 help? Perspectives on the development of neutralizing antibodies against SARS-CoV-2 Neutralizing and cross-reacting antibodies: 376 implications for immunotherapy and SARS-CoV-2 vaccine development 379 Differences in antibody kinetics and functionality between severe and mild SARS-CoV-380 2 infections Establishment and validation of a 382 pseudovirus neutralization assay for SARS-CoV-2 A SARS-CoV-2 385 serological assay to determine the presence of blocking antibodies that compete for 386 human ACE2 binding