key: cord-0891670-lmzssvv7 authors: Weber, Myriam C.; Risch, Martin; Thiel, Sarah L.; Grossmann, Kirsten; Nigg, Susanne; Wohlwend, Nadia; Lung, Thomas; Hillmann, Dorothea; Ritzler, Michael; Ferrara, Francesca; Bigler, Susanna; Egli, Konrad; Bodmer, Thomas; Imperiali, Mauro; Salimi, Yacir; Fleisch, Felix; Cusini, Alexia; Heer, Sonja; Renz, Harald; Paprotny, Matthias; Kohler, Philipp; Vernazza, Pietro; Risch, Lorenz; Kahlert, Christian R. title: Characteristics of Three Different Chemiluminescence Assays for Testing for SARS-CoV-2 Antibodies date: 2021-01-06 journal: Dis Markers DOI: 10.1155/2021/8810196 sha: 614355036ba0f777871139c68e630dfbcad1464c doc_id: 891670 cord_uid: lmzssvv7 Several tests based on chemiluminescence immunoassay techniques have become available to test for SARS-CoV-2 antibodies. There is currently insufficient data on serology assay performance beyond 35 days after symptoms onset. We aimed to evaluate SARS-CoV-2 antibody tests on three widely used platforms. A chemiluminescent microparticle immunoassay (CMIA; Abbott Diagnostics, USA), a luminescence immunoassay (LIA; Diasorin, Italy), and an electrochemiluminescence immunoassay (ECLIA; Roche Diagnostics, Switzerland) were investigated. In a multigroup study, sensitivity was assessed in a group of participants with confirmed SARS-CoV-2 (n = 145), whereas specificity was determined in two groups of participants without evidence of COVID-19 (i.e., healthy blood donors, n = 191, and healthcare workers, n = 1002). Receiver operating characteristic (ROC) curves, multilevel likelihood ratios (LR), and positive (PPV) and negative (NPV) predictive values were characterized. Finally, analytical specificity was characterized in samples with evidence of the Epstein–Barr virus (EBV) (n = 9), cytomegalovirus (CMV) (n = 7), and endemic common-cold coronavirus infections (n = 12) taken prior to the current SARS-CoV-2 pandemic. The diagnostic accuracy was comparable in all three assays (AUC 0.98). Using the manufacturers' cut-offs, the sensitivities were 90%, 95% confidence interval [84,94] (LIA), 93% [88,96] (CMIA), and 96% [91,98] (ECLIA). The specificities were 99.5% [98.9,99.8] (CMIA), 99.7% [99.3,99.9] (LIA), and 99.9% [99.5,99.98] (ECLIA). The LR at half of the manufacturers' cut-offs were 60 (CMIA), 82 (LIA), and 575 (ECLIA) for positive and 0.043 (CMIA) and 0.035 (LIA, ECLIA) for negative results. ECLIA had higher PPV at low pretest probabilities than CMIA and LIA. No interference with EBV or CMV infection was observed, whereas endemic coronavirus in some cases provided signals in LIA and/or CMIA. Although the diagnostic accuracy of the three investigated assays is comparable, their performance in low-prevalence settings is different. Introducing gray zones at half of the manufacturers' cut-offs is suggested, especially for orthogonal testing approaches that use a second assay for confirmation. COVID-19 is a recently emerging pandemic disease caused by infection with SARS-CoV-2 virus. Although there are considerable differences regarding its incidence, hospitalization rate, morbidity rate, and case fatality rate between different countries, disease control at the moment is uniformly achieved by strict prevention measures such as social distancing, wearing of face masks, hand-washing, contact tracing and testing, quarantine, or isolation [1] . Whereas the diagnosis of acute disease in the medical laboratory most importantly relies on RT-PCR testing, serological testing for antibodies specifically directed against viral proteins of SARS-CoV2 has increasingly come into the focus of public health authorities and medical institutions [2] [3] [4] [5] [6] [7] [8] . The clinical presentation of COVID-19 has been shown to be heterogeneous in severity as well as in clinical signs and symptoms [9] . A substantial proportion of patients have only minimal symptomatology or are even asymptomatic [10] . Currently, there is no vaccination available, and causal therapies are very limited [11] . After several countries showed success in confining the progression of the COVID-19 pandemic with drastic measures, there is a need to gradually cancel these measures to reinstitute possibly "normal" social and economic circumstances [12, 13] . To guide these actions to reverse socioeconomic lockdown, knowledge on the prevalence of COVID-19 is needed [14] . Since a large proportion of patients with suspected COVID-19 infection could not be tested by RT-PCR in the acute phase, serological testing may gain increasing importance for retrospective clarification of clinical symptoms [15] . Serologic testing allows us to estimate the proportion of individuals already infected with COVID-19, either in the total population, in healthcare workplace settings, or in general workplace settings [16, 17] . It facilitates contact tracing as well as surveillance and assists in the identification of individuals susceptible to COVID-19 infection [2, 18, 19] . Furthermore, individuals having had contact with confirmed COVID-19 patients might be interested in determining whether they developed SARS-CoV-2 specific antibodies, if they did not yet have access to testing. Moreover, serological testing allows us to clarify clinical cases in which RT-PCR testing has been negative despite a high pretest probability for the presence of COVID-19 [20, 21] . Such cases of false negative RT-PCR have been reported, possibly because of improper collection techniques, viral loads below the detectable limit of the assay, or diminished upper airway shedding of the virus [22] [23] [24] [25] . Finally, the role of specific antibodies in terms of protection against reinfection and persistence over time is currently not adequately defined. In the beginning of the pandemic, lateral flow tests were primarily employed to perform serological SARS-CoV2 testing [26] . However, some of these tests have been criticized for poor sensitivity and specificity [27] . Poor specificity has been suspected to occur due to cross-reactivity with the antibody response to endemic coronaviruses causing the common cold (i.e., HCoV-229E, -NL63, -OC43 and -HKU1) [28] . Some of these cross-reacting antibodies, however, have actually shown neutralizing activity against SARS-CoV-2 [29] . Since the antibody response in SARS-CoV-2 infection needs 2 to 4 weeks to develop, false negative antibody tests can occur due to insufficient duration between the onset of clinical symptoms and the time of blood sampling or insensitive measurement techniques that require large quantities of antibodies for a positive result [6, 12] . At the moment, it is not known how long the antibody titers persist. A recent Cochrane review stated that there are currently not enough studies available for antibody tests done after 35 days of symptom onset [30] . Whereas ELISA tests were available relatively early during the pandemic, the supply of these tests has been relatively scarce [31, 32] . Only recently have assays using chemiluminescence (CLIA) become available, with only a few validation data published so far [33, 34] . These assays are available in large quantities and can be analyzed in high-throughput settings. The different assays are directed against different specific antigens, i.e., the internal nucleocapsid (N) antigen or the surface spike protein (S1 or S1/S2) [27] . Not all SARS-CoV2 antibodies exhibit neutralizing properties. However, it has been shown that antibodies against nucleocapsid antigen and the receptor-binding domain (RBD) of the spike protein have a high correlation to virus neutralization titer [7] . The predictive values of negative and positive results depend on the pretest probability of the presence of COVID-19 disease. In low-prevalence settings (<5% seropositivity), positive predictive values are critically dependent on specificity [35] . Since the specificity of SARS-CoV2 antibody tests has been the subject of debate, false positive results can occur in low-prevalence settings [36] . False positive assay results are particularly problematic in low-prevalence settings, as these could lead concerned individuals into a false sense of security and increase their risk of contracting COVID-19 disease through unsafe behavior [13] . We aimed to investigate the diagnostic specificity and sensitivity of newly released chemiluminescence immunoassays (CLIAs). We investigated these assay formats in a cohort of confirmed COVID-19 patients to assess diagnostic sensitivity. Subsequently, we assessed the diagnostic specificity of 2 Disease Markers these tests in a cohort of healthcare workers and a cohort of healthy blood donors. Finally, we assessed the analytical specificity of the different assays in samples collected prior to the COVID-19 pandemic, which may potentially contain cross-reacting antibodies. In a model, we finally related the identified diagnostic characteristics to negative and positive predictive values depending on the pretest probability of having had COVID-19 in the personal history. Population. This is a study on diagnostic tests used for the detection of SARS-CoV-2 antibodies in individuals originating from Switzerland and Liechtenstein. Anonymized samples originating from four different types of patients were investigated with three CLIAs. The first group of patients was retrospectively assembled from Liechtenstein and Swiss patients having their laboratory evaluations sent to labormedizinisches zentrum Dr. Risch in Vaduz (Liechtenstein) and Buchs (Switzerland) and consisted of COVID-19 patients (mainly outpatients) whose serum was drawn after COVID-19 disease was confirmed by RT-PCR between March 2nd and April 23rd, 2020 (n = 145). This group was used for the determination of sensitivity. The method of RT-PCR measurement has been conducted as reported elsewhere [37] . The patients from Liechtenstein were prospectively and consecutively enrolled within a national COVID-19 cohort, whereas the Swiss patients consisted from a retrospectively assembled convenience sample, both detailed in references [37, 38] . The second group of individuals was prospectively assembled and consisted of consecutive healthy blood donors from the Blutspendedienst Graubünden without clinical suspicion of COVID-19 providing blood for testing SARS-CoV2 antibodies from April 15th to May 4th, 2020 (n = 191). In addition to fulfilling the normal criteria of blood donation, these blood donors specifically responded they had had no had flu-like symptoms or contact with a known COVID-19 patient during the past 14 days. The third cohort was prospectively assembled within a study setting from the Kantonsspital St. Gallen and consisted of healthcare workers providing a blood sample for detecting SARS-CoV2 antibodies between March 19th and April 3rd, 2020 (n = 1002) as described elsewhere [39] . Hospital admissions in this region peaked in the second week of April 2020. The blood donors and the healthcare workers were used to determine specificity. Since these 2 groups did not have RT-PCR testing available, samples with at least two of the three chemiluminescence assays positive were excluded from this analysis, assuming that these individuals had occult SARS-CoV-2 infection. Such an orthogonal testing approach for SARS-CoV-2 antibodies has been reported to have a very high positive predictive value for COVID-19 and therefore is suited to reliably exclude individuals with recent COVID-19 infection [40] [41] [42] . Characteristics of the excluded 10 seropositive individuals among the healthcare workers are detailed elsewhere (i.e., 6 individuals had all three assays positive, 3 had CMIA and ECLIA positive, and one had CMIA and LIA positive) [ Whereas the cohort with healthcare workers provided written informed consent, informed consent for performing laboratory analysis on anonymized samples in the other three groups was waived. The study was taking into account the STARD guidelines [43] . For each serum sample, the age and sex of the individual, the type of clinical setting, and the number of days after first positive RT-PCR it was drawn (if applicable) were available. The sera employed for testing were either fresh or stored at -25°C for less than 18 months. The antibodies were tested on the following diagnostic platforms: COBAS 6000 (Roche Diagnostics, Rotkreuz, Switzerland), Abbott Architect i2000 (Abbott Diagnostics Baar, Switzerland), and Liaison XL (Diasorin, Luzern, Switzerland). The Roche Diagnostics assay (Elecsys® Anti-SARS-CoV-2; ECLIA) employs a recombinantly engineered nucleocapsid antigen for the detection of total immunoglobulin. The molecular target from the Abbott Diagnostics assay (SARS-CoV-2 IgG; CMIA) is also the nucleocapsid antigen, and it measures specific IgG levels. The Diasorin assay (LIAISON® SARS-CoV-2 S1/S2 IgG; LIA) measures specific IgG directed against S1/S2 antigens. To further elucidate the effects of any cross-reacting antibodies in the 3 chemiluminescence assays in the samples with EBV, CMV, or following endemic coronavirus, we also employed the Euroimmun SARS-CoV-2 ELISA (Euroimmun, Luzern, Switzerland) to measure specific IgG and IgA on a DSX instrument (Dynex Technologies, Denkendorf, Germany). The coefficients of variation (CV's) of the employed methods in our hands were 2.7% for ECLIA, 3.6% for CMIA, 5.4% for LIA, 4.6% for IgG ELISA, and 3.6% for IgA ELISA. 3 Disease Markers 2.3. Statistical Methods. Specificity was determined in the samples originating from blood donors and healthcare workers. The sensitivity of the different assays was assessed in the group of patients with RT-PCR confirmed COVID-19 disease. The following cut-offs provided by the manufacturers were employed: a COI ðcut − off indexÞ > 0:9 for ECLIA, a S/C value ðextinction of the patient sample divided by the extinction of the calibratorÞ > 12 for LIA, a S/C value ≥ 1:4 for CMIA, and a S/C value of ≥1.1 in the ELISA. We also applied alternative cut-offs (half and double of the manufacturers' cut-offs), in order to better understand the relationship between signal strength and associated diagnostic characteristics. Positive and negative likelihood ratios (LR) were calculated for the different cut-off levels [44] . Regarding interpretation of LR, it is generally acknowledged that a + LR > 10 as well as a -LR < 0:1 generates large and often conclusive changes from the pretest to the posttest probability. A +LR between 5 and 10 as well as a -LR between 0.1 and 0.2 generates moderate shifts from the pretest to the posttest probability, whereas a +LR between 2 and 5 and a -LR from 0.2 to 0.5 generate small but sometimes important changes from the pretest to the posttest probability [44] . Receiver operating characteristic (ROC) curves with the area under the curves (AUCs) were calculated as an indicator of diagnostic accuracy. The AUCs for the different parameters were compared by the method of Hanley and McNeil. Positive and negative predictive values for each of the employed assays were then plotted as a function of pretest probability, as described earlier [45] . Continuous variables are given as medians and interquartile ranges (IQRs), whereas proportions are given as percentages together with 95% confidence intervals (CIs). Agreement between the two methods was assessed by Cohen's kappa. The associations between variables were calculated with Spearman's rank correlation. The group of RT-PCR confirmed COVID-19 patients consisted of 145 individuals with a median age of 46 years (IQR [30, 58] years), and 79 of the patients, i.e., 48% (95% CI [40, 56] ), were female. The sera were taken after a median of 47 IQR ( [40, 54] ; minimum 21, maximum 66) days after first presentation with suspected COVID-19. The blood donors had a median age of 44 years IQR [28, 53] , and 90 of the 191 donors, i.e., 47% (95% CI [40, 54] ) were female. In the cohort of healthcare workers, the median age was 38 years (IQR [30, 49] The correlation between the 3 different assays was highly significant (p < 0:0001). The agreement of the methods when we used the manufacturers' cut-offs showed a kappa value of 0.92 between CMIA and LIA, 0.96 between CMIA and ECLIA, and 0.94 between LIA and ECLIA. In the COVID-19 patients, there was a significant inverse association between days after RT-PCR and S/C in CMIA (r = −0:25, p = 0:003), but not in LIA or ECLIA. 3.2. Diagnostic Specificity and Sensitivity at the Manufacturers' Cut-Offs. When looking at specificities of the different assays at the cut-offs provided by the manufacturers within the group of blood donors and healthcare workers without COVID- 19 3.3. Multilevel Likelihood Ratios. We then calculated multilevel likelihood ratios at the manufacturers' cut-off levels as well as at half and double the manufacturers' cut-off levels, i.e., a S/C of 0.7, 1.4, and 2.8 for the CMIA, respectively, a S/C of 6, 12, and 24 for the LIA, and a COI of 0.5, 1, and 2 for the ECLIA. Table 1 illustrates the different positive (+LR) and negative likelihood ratios (-LR). Regarding diagnostic value of the LR, the investigated serological tests can be considered to offer meaningful diagnostic characteristics. Of note, clinical characteristics are already meaningful at lower cut-off levels than recommended by the manufacturer. Characteristics. The positive predictive values (PPVs) for three different cut-offs (manufacturers' cut-offs and half as well as double the manufacturers' cutoff values) are shown in Figures 2(a) (CMIA), 2(b) (LIA), and 2(c) (ECLIA). It can be seen that ECLIA, especially in low pretest probability settings, due to the high specificity has somewhat higher PPVs than the other assays: at the manufacturer's cut-off, the PPV was 97% at a pretest probability of 3 percent, whereas at half of the manufacturer's cut-off, the PPV at a pretest probability of 3% was 95%. For the LIA, the PPV at a pretest probability of 5% was 95%, whereas the same PPV at half of the manufacturer's cut-off was achieved with a pretest probability of 18%. At double the manufacturer's cut-off, a PPV of 95% in the LIA was achieved with a pretest probability of 2%. Finally, with the CMIA, at the manufacturer's cut-off, a PPV of 95% was achieved at a pretest probability of 5%, whereas at half of the manufacturer's cut-off, the same PPV was seen at a pretest probability of 23%. At double the manufacturer's cut-off, the CMIA had a PPV of 95% at a pretest probability of 4%. Regarding negative predictive values (NPVs; Figure 3 ), the ECLIA due to the highest sensitivity had a NPV of 99% up to a pretest probability of 26% at the manufacturer's cut-off. At half of the manufacturer's cut-off, a NPV of 99% was seen up to a pretest probability of 30%. With the LIA, the NPV at the manufacturers' cut-off was at 99% up to a pre-test probability of 11%, whereas at half of the manufacturer's cut-off, the NPV was at 99% up to a pretest probability of 30%. In the CMIA, the NPV at the manufacturer's cut-off was 99% up to a pretest probability of 21%, whereas at half of the cut-off, the NPV was 99% up to a pretest probability of 26%. 3.5. Analytical Specificity. None of the assays showed positive antibodies in sera from patients with active EBV or CMV disease. Of the sera taken after endemic coronavirus infection, 4 were infected with RC229E, 3 were infected with RCNL63, 2 were infected with RCHKU1, 2 had an infection with RCOC43, and one patient had both RC229E and RCNL63. , whereas the other samples did not reveal a detectable signal. The sample of a patient with RCOC43 infection with a detectable CMIA signal also had a detectable LIA signal of 6.1, whereas all other samples had unmeasurable signals. Altogether, even if SARS-CoV-2 antibody titers following endemic coronavirus infections were below the manufacturers' cut-offs for positivity, 3 samples exhibited detectable antibodies in the CMIA assay, one of which was also in the LIA assay. The present study investigated the analytical (analytical specificity), diagnostic (diagnostic sensitivity and specificity), and operational characteristics (likelihood ratios, predictive values) of three highly automated assays for the detection of antibodies against SARS-CoV-2. These characteristics were investigated after a median of 47 days after PCRpositivity. Although the three test formats showed comparable diagnostic accuracy, test performance varied regarding positive predictive values in the low-prevalence setting. Finally, the investigated tests did not show interference with two commonly encountered infections, i.e., EBV and CMV. There was, however, cross-reactivity in some patients with proven previous infection by endemic coronaviruses for two of the three analyzed tests. The timing of antibody development in COVID-19 disease is crucial. Long and colleagues demonstrated that IgG or IgM can be detectable in approximately 60% of patients at days 5-7 after symptom onset, in 95% of patients after 12-14 days and in 100% of patients after 17 to 19 days [6] . Tang and colleagues demonstrated a seroconversion rate of 93.8% for CMIA and 89.4% for ECLIA 14 days after symptom onset and later [46, 47] . A systematic review performed by the Cochrane collaboration regarding the usefulness of antibody tests to diagnose current or past infection with SARS-CoV-2 stated that there are too few studies available to estimate test sensitivity beyond 35 days after symptom onset [30] . The present study, with a median of 47 days after presentation for symptomatic COVID-19 disease, therefore fills an important gap. Laboratory results are interpreted by combining pretest probabilities with diagnostic characteristics to obtain negative and positive predictive values according to Bayes' theorem [48] . To the best of our knowledge, there is no clinical score available to predict the pretest probability of COVID-19 based only on clinical symptoms and history without any laboratory or imaging results [49] . Such a score based on clinical history taking would improve the interpretation of COVID-19 serology results, not least in a retrospective setting, where accurate laboratory results are not available because no blood has been drawn. However, there are several other proxies to assess pretest probabilities. In the Swiss and Liechtenstein setting, several situations can be described, in which the risk for patients to contract COVID-19 can be quantified [37] : 32% for household contacts, 13% of close working contacts, and 12-25% of patients with fever > 38°C and respiratory symptoms. Knowing the temporal and 9 Disease Markers regional seroprevalence of the region also allows for an approximation of pretest probability even without knowing the clinical symptoms of a specific patient [50] . The predictive values depend not only on the pretest probability but also on the sensitivity and specificity of a test at a given cut-off. Negative results using the manufacturers' cut-offs had an NPV of 99% up to a pretest probability of 11% to 26%. At half of the manufacturer's cut-off, past COVID-19 could be ruled out with a probability of 99% up to a pretest probability of 26% (CMIA) to 30% (LIA, ECLIA). These pretest probabilities are in the range of those observed in symptomatic individuals. Thus, serology done in patients with clinical symptoms three or more weeks before drawing blood together with manufacturers' and modified manufacturers' cut-offs can be considered safe to exclude COVID-19 disease. Nevertheless, serological nonresponders can occur with a frequency of approximately 3% and may go overlooked. False negative results, if they occur at a low frequency, can be considered of minor importance because they are not expected to change an individual's behavior in response to its potential damage. Regarding PPV, which is the probability that a positive result indicates a past infection with SARS-CoV-2, the ECLIA with the manufacturer cut-off provided the best operational characteristics at low pretest probabilities ≤ 5%: In this assay, the PPV at 3% pretest probability was 97%, which raises the question whether positive ECLIA results should be confirmed by an orthogonal testing approach [40, 51, 52] . When taking the lower limit of the 95% CI of the specificity, i.e., 99.5%, the PPV at 1% pretest probability is 66%, and 91% at 5% pretest probability. We therefore recommend continuing with the confirmation of positive ECLIA results with a second assay [40] . Our data also show that at the manufacturer cut-offs, the CMIA and LIA need confirmatory testing in low pretest probability (<5%) settings. Otherwise, the risk of false positive results is larger than 5%. The problem of false positive results is that they can change the behavior of individuals in that they will not adhere to hygienic and distancing measures after the (false) positive result has been disclosed. This puts individuals at increased risk for contracting COVID-19, with its potentially devastating consequences [13] . The medical laboratory thus has a great responsibility to deliver meaningful results to the patients and their physicians. Common recommendations on likelihood ratios state that +LR > 10 and −LR < 0:1 generate meaningful changes from pretest to posttest probability [44] . Our findings suggest that all three assays provide meaningful information, even at half of the manufacturers' cut-off values. The figures of pretest probabilities against NPV illustrate that taking half of the manufacturer cut-off as a decision limit will still safely rule out COVID-19: a cut-off as low as half of the manufacturer cut-off will correctly detect 98.9% ( Looking at the relationship between pretest probability and PPV at a cut-off half of the manufacturer's cut-off illustrates that such a cut-off does not reliably diagnose past COVID-19 in low test probability settings. However, using an orthogonal testing approach recommended by the U.S. Food and Drug Administration [41] , i.e., an ECLIA result Disease Markers of 0.6 together with an LIA result of 6.1, would lead to a PPV of 99.1% at a pretest probability of 1%. A similar result would be achieved with a CMIA result of 0.7 together with an LIA result of 6.1, which at a pretest probability of 2% results in a PPV of 99%. This example illustrates that the introduction of gray zones should be considered not only for ELISA but also for chemiluminescence immunoassays when offering SARS-CoV-2 testing to clinicians and public health professionals [53] . The fact that we found some cross-reactivity in patients with prior endemic coronaviruses raises the question whether this finding represents analytical cross-reactivity or reflects cross-reactive immunity conferred by recent endemic coronavirus disease. We did not conduct neutralization assays to clarify this issue. However, Ng and colleagues reported that patient sera from human coronaviruses variably reacted with SARS-CoV-2 S-antigen and nucleocapsid antigen, but not with the S1 subunit [29] . These patient sera exhibited neutralizing activity against SARS-CoV-2 S pseudotypes according to the levels of SARS-CoV-2 S-binding IgG and with efficiencies comparable to those of COVID-19 patient sera [29] . In our study, we had one case with a LIA signal (targeting S1-S2 antigen) and a CMIA signal (targeting nucleocapsid antigen) without a signal in the IgG ELISA (which is targeted against S1). This patient had his serum taken in September 2019 after infection with RCOC43 was diagnosed in January 2019. We are convinced that identifying patients with endemic coronaviruses with potential neutralizing cross-reactivity will be of increasing importance in the future. Whereas CMIA and LIA could play a role in identifying such individuals, ECLIA is not expected to help further in this aspect, as none of the patients with endemic coronavirus had a measurable antibody response. The measurements in the healthcare workers as well as in the blood donors suggest that such constellations do not occur often: 1147/1193 (i.e., 96.1 (95% CI [94.9,97.1])) have unmeasurable antibody titers in the LIA, whereas this frequency amounts to 921/1193 (i.e., 77.2% (95% CI [74.7,79.5])) in the CMIA. Our study has strengths and limitations. A strength is that specificity has been assessed in a large group of 1193 individuals without evidence of COVID-19. Such an approach offers the possibility to describe the specificity with relatively narrow confidence intervals. A further strength is that we investigate several potential cut-offs for clinical decision making. A limitation of the study is that samples employed for evaluation of specificity were selected from contemporary and not prepandemic participants. Two positive serology results in an orthogonal testing approach cannot provide 100% certainty that any remaining false positives are truly false positive. Nevertheless, we demonstrate that a combination of two positive results with chemiluminescence assays has a very high positive predictive value even at low pretest probabilities, comparable to that known from molecular methods [42] . Further, inclusion of the index serology tests as part of the reference standard definition of absence of disease carries a risk of bias in results. At low pretest probabilities, such a bias might have a considerable impact on positive predictive values. There are several factors mitigating such a misinterpretation: (a) utilizing an orthogonal testing approach in initially positive results, (b) the fact that even in the first wave of COVID-19, COVID-19 prevalence in some regions of Europe was already more than 10%, which cannot be considered low [50] . The specificities identified in our study from contemporary samples are comparable to those obtained in a similar study on prepandemic samples [54] . The study is finally limited by the fact that we did not have a full clinical description of the COVID-19 patients included in this study, and patients hospitalized due to COVID-19 represented a minority of the COVID-19 patients. However, since the majority of patients subjected to serology will originate from the outpatient setting, we believe that this fact actually strengthens our findings. In sum, we think that the limitations do not invalidate our findings. In conclusion, we evaluated the serology in patients at a median of 47 days following the first presentation of suspected COVID-19 infection and selected individuals without COVID-19 during the pandemic. We found that the diagnostic accuracy of the three investigated assays was comparable. Assay cut-offs have not been designed for orthogonal testing. Introducing gray zones at half of the manufacturers' cut-offs is suggested. These differentiated cut-offs would allow for safer ruling out or ruling in of past COVID-19 infections. Such an approach would allow us to more appropriately select samples for further testing in an independent assay in an orthogonal testing algorithm. We propose that our findings be replicated in other populations. The data used to support the findings in this study will be available from the corresponding author upon request. The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analysis, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results. Myriam Weber was responsible for conceptualization, methodology, formal analysis, and writing-original draft. Martin Risch was responsible for conceptualization, methodology, formal analysis, and writing-original draft. Sarah Thiel was responsible for data curation, validation, and resources. Kirsten Grossmann was responsible for resources. Susanne Nigg was responsible for resources. Kirsten Nadja Wohlwend was responsible for investigation, validation, and resources. Thomas Lung was responsible for validation and resources. Dorothea Hillmann was responsible for investigation, validation, project administration, and resources. Michael Ritzler was responsible for resources. Francesca Ferrara was responsible for resources. Susanna Bigler was responsible for validation and resources. Konrad Egli was responsible for validation, resources. Thomas Bodmer was responsible for 11 Disease Markers validation and resources. Mauro Imperiali was responsible for validation and resources. Yacir Salimi was responsible for validation and resources. Felix Fleisch was responsible for resources. Alexia Cusini was responsible for resources. Sonja Heer was responsible for resources. Harald Renz was responsible for conceptualization. Matthias Paprotny was responsible for supervision, resources, and writing-original draft. Philipp Kohler was responsible for validation and writing-original draft. Pietro Vernazza was responsible for conceptualization, funding acquisition, and writing-original draft. Lorenz Risch was responsible for conceptualization, methodology, funding acquisition, supervision, resources, and writing-original draft. Christian R. Kahlert was responsible for conceptualization, methodology supervision, resources, and writing-original draft. All authors were responsible for writing-reviewing and editing. Myriam C. Weber and Martin Risch contributed equally to this manuscript. The COVID-19 pandemic: a comprehensive review of taxonomy, genetics, epidemiology, diagnosis, treatment, and control Report from the American Society for Microbiology COVID-19 International Summit Should RT-PCR be considered a gold standard in the diagnosis of COVID-19? Role of serology in the COVID-19 pandemic Early detection of SARS-CoV-2 antibodies in COVID-19 patients as a serologic marker of infection Antibody responses to SARS-CoV-2 in patients with COVID-19 Temporal profiles of viral load in posterior oropharyngeal saliva samples and serum antibody responses during infection by SARS-CoV-2: an observational cohort study The role of antibody testing for SARS-CoV-2: is there one? Clinical characteristics of coronavirus disease 2019 in China Asymptomatic cases with SARS-CoV-2 infection Antiviral therapy in management of COVID-19: a systematic review on current evidence A phased approach to unlocking during the COVID-19 pandemic-lessons from trend analysis Waiting for certainty on Covid-19 antibody tests -at what cost? Audio interview: the challenges of safe reopening Covid-19: lack of capacity led to halting of community testing in March, admits deputy chief medical officer The important role of serology for COVID-19 control Knowledge and perceptions of COVID-19 among health care workers: cross-sectional study Investigation of three clusters of COVID-19 in Singapore: implications for surveillance and response measures Connecting clusters of COVID-19: an epidemiological and serological investigation Serological immunochromatographic approach in diagnosis with SARS-CoV-2 infected COVID-19 patients Profiling early humoral response to diagnose novel coronavirus disease (COVID-19) Diagnostic performance of CT and reverse transcriptase-polymerase chain reaction for coronavirus disease 2019: a meta-analysis A cautionary tale of false-negative nasopharyngeal COVID-19 testing Diagnosis of the coronavirus disease (COVID-19): rRT-PCR or CT? Potential preanalytical and analytical vulnerabilities in the laboratory diagnosis of coronavirus disease 2019 (COVID-19) Covid-19: testing times Will antibody tests for the coronavirus really change everything? Serological assays for emerging coronaviruses: challenges and pitfalls Pre-existing and de novo humoral immunity to SARS-CoV-2 in humans Antibody tests for identification of current and past infection with SARS-CoV-2 Severe acute respiratory syndrome coronavirus 2-specific antibody responses in coronavirus disease patients In vitro diagnostic assays for COVID-19: recent advances and emerging trends Analytical performances of a chemiluminescence immunoassay for SARS-CoV-2 IgM/IgG and antibody kinetics Performance characteristics of the Abbott architect SARS-CoV-2 IgG assay and seroprevalence in Antibody detection and dynamic characteristics in patients with COVID-19 Antibody tests for COVID-19: drawing attention to the importance of analytical specificity Flattening the curve in 52 days: characterization of the COVID-19 pandemic in the Principality of Liechtenstein Frequency of serological non-responders and false-negative RT-PCR results in SARS-CoV-2 testing: a population-based study Prevalence of SARS-CoV-2 antibodies among Swiss Hospital workersresults of a prospective cohort study Interim Guidelines for COVID-19 Antibody Testing EUA authorized serology test performance Temporal course of SARS-CoV-2 antibody positivity in patients with COVID-19 following the first clinical presentation STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies Users' guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? The Evidence-Based Medicine Working Group The predictive characteristics of D-dimer testing in outpatients with suspected venous thromboembolism: a Bayesian approach Clinical performance of two SARS-CoV-2 serologic assays Clinical performance of the Roche SARS-CoV-2 serologic assay Diagnostic tests: a statistical review A novel risk score to predict diagnosis with coronavirus disease 2019 (COVID-19) in suspected patients: a retrospective, multicenter, and observational study SEROCoV-POP): a population-based study COVID-19 diagnostics, tools, and prevention Molecular and serological tests for COVID-19 a comparative review of SARS-CoV-2 coronavirus laboratory and point-of-care diagnostics Validation of a commercially available SARS-CoV-2 serological immunoassay Sideby-Side comparison of three fully automated SARS-CoV-2 antibody assays with a focus on specificity The research project was funded by a grant from the government of the Principality of Liechtenstein and the Swiss National Science Foundation (Project ID 196544). The help of Toni Schönenberger and Walter Frehner in identifying samples is acknowledged.