key: cord-0870904-5h64q2bo
authors: Yang, Jianbo; Pederson, Edward C; Hamilton, Christopher; Neibauer, Terri; Robyak, Kimberly; McGhee, Pamela; Speicher, Teresa; Zhu, Yusheng
title: Analytical and Clinical Analysis of Two Automated Anti-SARS-CoV-2 Immunoassays in Pre-pandemic and Pandemic Patient Populations
date: 2020-11-05
journal: J Appl Lab Med
DOI: 10.1093/jalm/jfaa204
sha: bc8f3574055638fd7c3de6e03d993f5d3c56a569
doc_id: 870904
cord_uid: 5h64q2bo

BACKGROUND: In the absence of a safe, effective vaccine, the worldwide spread of COVID-19 (SARS-CoV-2) infection will continue. Laboratory tests with ideal precision, sensitivity, and specificity should be used in public health and clinical settings to gauge the extent of virus exposure. Toward this end, we evaluated the analytical and clinical performance of the Abbott SARS-CoV-2 IgG and the Roche Anti-SARS-CoV-2 immunoassays. METHODS: Quality control, pooled COVID-19, and non-COVID-19 patient specimens were used for the imprecision study. 246 specimens from 70 patients with COVID-19 diagnosis were tested to study the sensitivity. 73 non-COVID-19 control specimens were measured to study the specificity. All specimens were analyzed by both assays. RESULTS: Total analytic variability (CV) of the negative and positive controls were 5.5% and 3.6% for the Abbott assay and 4.5% and 1.9% for the Roche assay. Both assays demonstrated 100% qualitative reproducibility of negative and positive controls. The clinical specificities of the Abbott and the Roche assays were 100% (95% CI: 94% - 100%) and 97% (95% CI: 90% - 100%), respectively. The clinical sensitivities of the Abbott assay were 49% (95% CI: 41% - 56%), 86% (95% CI: 74% - 93%) and 100% (95% CI: 76% - 100%) for samples collected at 0-6 days, 7-13 days and ≥14 days after the first RT-PCR, while the sensitivities of the Roche assay were 55% (95% CI: 47% - 62%), 86% (95% CI: 74% - 93%) and 100% (95% CI: 76% - 100%). CONCLUSIONS: This study demonstrates similar analytical and clinical performance of the Abbott and the Roche SARS-CoV-2 antibody assays, but the Roche assay may be slightly more sensitive for patients tested within 0-6 days after first positive RT-PCR of SARS-CoV-2.

Coronavirus disease 2019 (COVID-19) is a respiratory infectious disease caused by the novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), a single-stranded and enveloped RNA virus of the family Coronaviridae (1) . COVID-19 can cause severe respiratory illness with high mortality in patients with comorbidities such as diabetes, hypertension and cardiovascular disease (2) . Laboratory testing plays an important role in identifying infected patients and providing pandemic information for the government to make proper strategies to alleviate or stop the spread of this virus. Nucleic acid based diagnostic molecular testing, primarily reverse transcription polymerase reaction (RT-PCR) for detecting SARS-CoV-2 RNA, usually requires nasopharyngeal, oropharyngeal, or nasal swab samples, and false-negative results can occur if specimens are collected inappropriately (3) . Combining RNA and antibody testing significantly improves the sensitivity of pathogenic diagnosis for COVID-19 in the early phase of infection (4) . A study shows the RT-PCR positivity in patients with mild symptoms starts to decline by week 3 after the disease onset and subsequently becomes undetectable (5) . Serological diagnosis is potentially important for patients with mild to moderate illness who may present late, beyond the first 2 weeks of illness onset. Serological diagnosis is also becoming an crucial tool for understanding the extent of COVID-19 in the community and to identify individuals who are already exposed and potentially "protected" from a second infection (6) .

Although nucleic acid based diagnostic methods are well known for its high sensitivity and specificity, they also have disadvantage of dependence on either specialized instruments or specially trained laboratory professionals to run the test, therefore not readily available to some hospital laboratories. This may cause longer turnaround time and adversely impact the management of certain patients (7) . Serology tests for detecting the virus antibodies could be implemented on automated immunoassay analyzers that are available to most hospital laboratories in the United States. Several diagnostic manufacturers have released automated serology assays for SARS-CoV-2 antibody testing.

Studies have already showed that immunocompetent patients infected with SARS-CoV-2 may develop IgA, IgM and/or IgG antibodies against SARS-CoV-2 (8) (9) (10) .

Most SARS-CoV-2 immunoassays have similar principle employing the spike protein, nucleocapsid protein, or receptor binding domain as the bait to detect the corresponding IgG or total antibodies. However, their clinical performance may vary, as shown by a study on four commercial SARS-CoV-2 IgG immunoassays (11) . Both the Abbott SARS-CoV-2 IgG and the Roche Anti-SARS-CoV-2 assay are sandwich immunoassays utilizing microparticles coated with the nucleocapsid protein antigens as the capturing molecule, although it is unknown whether the same epitopes are used. The Abbott assay employs anti-human IgG acridinium-labeled conjugate as the detecting molecule to detect the IgG antibodies against the nucleocapsid protein, whereas the Roche assay utilizes the ruthenium-labeled nucleocapsid protein antigen as the detecting molecule to detect the total antibodies, presumably including IgA, IgM and IgG. In this study, we compared the analytical and clinical performance of the Abbott SARS-CoV-2 IgG and the Roche Anti-SARS-CoV-2 assay.

This investigation was approved by the Institutional Review Board of Pennsylvania State University Hershey Medical Center (study number STUDY00015728). We analyzed imprecision, sensitivity and specificity in different specimen types of various patient populations including pre-pandemic and pandemic patients. 

For the Abbott assay, both negative QC (mean S/C index: 0.06) and positive QC samples (mean S/C index: 3.15) were analyzed 38 times on two Architect i1000 analyzers on 7 different days. The total CVs were 5.5% and 3.6% for negative and positive QC. The reproducibility of the results of both QC samples was 100% (95% CI: 89% -100%). For the Roche assay, the pooled COVID-19 negative (mean COI: 0.076) and positive samples (mean COI: 8.59) were analyzed 72 times and 40 times, respectively, by four Roche Cobas e602 analyzers on 7 different days. The total CVs were 4.5% and 1.9% for the negative and positive pooled samples. The reproducibility was 100% (95% CI: 94% -100%) for the negative and 100% (95% CI: 89% -100%) for the positive pooled samples.

There were 139 pairs of heparin and EDTA plasma collected from the same draw, including 104 pairs from COVID-19 patients and another 35 pairs from non-COVID-19

patients. Both the Roche Anti-SARS-CoV-2 and the Abbott SARS-CoV-2 IgG assays showed 100% (95% CI: 97% -100%) concordance between the results from the paired heparin plasma and EDTA plasma samples.

Both the Roche and the Abbott assays were non-reactive in the 5 samples collected from non-COVID-19 patients with other respiratory pathogen infections and 30 randomly selected non-COVID-19 patients. For the 38 samples collected before this COVID-19 pandemic, the Abbott assay results were all non-reactive, but the Roche assay generated 2 reactive results with the COI values of 1.18 and 5.01, respectively. Therefore, the specificities of the Abbott and the Roche assays were 100% (95% CI: 94% -100%) and 97% (95% CI: 90% -100%), respectively. The difference in specificities between these two assays was not statistically different (p = 0.500).

Sensitivity 246 samples were collected on different days from 70 patients who were diagnosed with COVID-19 by the RT-PCR test. The Roche assay generated 160 reactive results and the sensitivity was 65% (95% CI: 59%-71%). The Abbott assay identified 149 reactive samples and the sensitivity was 61% (95% CI: 54%-67%) ( Fig. 1 and Table 1 ). Only 15 samples from 13 patients showed discordant results between the Roche and the Abbott assays, with 13 samples reactive in the Roche assay but non-reactive in the Abbott assay and vice versa for the other 2 samples. These 13 patients became serologically positive by both assays later, and the 15 samples with discordance were collected around the seroconversion periods. For all 246 samples, the agreement between these two assays was 94% (95% CI: 90% -96%). The McNemar chi-square test suggested the discordance was statistically significant (p = 0.005).

These 246 samples were further classified according to blood collection dates after their first positive SARS-CoV-2 RT-PCR results. 173 samples were collected within 6 days, 57 samples between 7 to 13 days, and 16 samples 14 days or later after the first positive RT-PCR test. The Roche assay showed sensitivities of 55% (95% CI: 47% -62%), 86% (95% CI: 74% -93%), and 100% (95% CI:76% -100%) in these three groups of samples, whereas the Abbott assay demonstrated sensitivities of 49% (95% CI: 41% -56%), 86% (95% CI: 74% -93%) and 100% (95% CI:76% -100%). For the same 3 groups of specimens, the agreement between these two assays were 91% (95% CI: 86% -95%), 100% (95% CI: 94% -100%), and 100% (95% CI: 81% -100%). The McNemar chi-square test analysis suggested that the Roche assay detected more positive results than the Abbott assay in the specimens collected within 6 days after the first positive RT-PCR test (p = 0.005). However, for samples collected between 7 and 13 days and ≥ 14 days, there was no difference in sensitivity between these two assays.

According to the manufacturers' package inserts, the Abbott SARS-CoV-2 IgG assay has a specificity of 99.6% and the Roche Anti-SARS-CoV-2 assay has a specificity of 99.81%. The specificities for the Roche and the Abbott assays in this study of 73 specimens were 97% (95% CI: 90% -100%) and 100% (95% CI: 94% -100%), respectively, similar to the manufacturers' claims. The COI values of the 2 reactive pre-pandemic samples measured by the Roche assay were 1.18 and 5.01. The COI value of 1.18 is very close to the cutoff value of 1.0. This could be due to a random error for a sample in the borderline zone and may become negative by retesting. The COI of the second false-positive sample was 5.01. This could be due to an interference or the cross reactivity with other coronavirus strains.

The patient had hospital-acquired pneumonia, but it was unknown if the patient was infected with other coronavirus strains. This sample was also assayed by the DiaSorin SARS-CoV-2 S1/S2 IgG assay on the LIAISON XL analyzer (DiaSorin, Stillwater, MN, USA) and the result remained negative. The Roche package insert states that this assay has no biotin interference in serum concentrations up to 1200 ng/mL. Studies also showed no interference from hemolysis, icterus, lipemia (13) , and rheumatoid factors (14) . According to the package insert, potential interferences from pharmaceutical compounds other than biotin have not been tested and an interference cannot be excluded. In addition, in rare cases, interference due to extremely high titers of antibodies to analyte-specific antibodies, streptavidin or ruthenium can occur. We should point out that the prepandemic, respiratory pathogen-positive, and pandemic samples used for the specificity study were from presumptive non-COVID-19 patients because they were not confirmed by the RT-PCR testing. This is a limitation of the study, although the pre-pandemic samples were unlikely from individuals with SARS-CoV-2 infection. Another limitation of this study is that we did not use a QC sample with signal index close to the cutoff for the imprecision study. The manufacturers should provide this kind of QC samples in the future.

According to the CDC interim guidelines for COVID-19 antibody testing, the prevalence of SARS-CoV-2 antibody is expected to be low, ranging from <5% to 25%, so that testing at this point might result in relatively more false-positive results and fewer false-negative .0%) at 0-6 days, 7-13 days, and ≥ 14 days, after RT-PCR confirmation. In our data set, the sensitivity of the Roche assay (55%) in samples collected within 0-6 days post RT-PCR is higher than that of the Abbott assay (49%). As the confidence intervals for the sensitivity of the two assays, overlap, the clinical significance of this finding remains unclear. This observation may be due to differing definitions of early disease such as "after symptom onset" or "post RT-PCR" as shown in other studies (14) (15) (16) . RT-PCR testing is typically performed after symptom onset and the delay may vary widely. Tang et al also reported that the sensitivity of the Roche assay is higher when time from positive RT-PCR (0-13 days) instead of time from symptom onset was used as the starting point (14) . We found that the sensitivities of both Abbott and Roche assays were 86% and 100% in samples collected between 7 and 13 days and ≥14 days post RT-PCR confirmation. It was reported that the sensitivities of the Abbott and the Roche assays in samples collected 13 over 14 days post symptom onset were 93.75% and 89.36% (13, 15) . Another recent study reported that the Abbott SARS-CoV-2 IgG assay had a specificity of 99.9% from 1020 pre-COVID-19 samples and a sensitivity of 100% at 17 days after symptom onset or 13 days after PCR positivity (17) . The data in the current study additionally suggest that the cutoff values may dictate differing sensitivity and specificity. As shown in Fig.1 , the S/C indices of non-COVID-19 patients in the Abbott assay are well below the cutoff value of 1.4.

However, our study only included 73 non-COVID-19 samples. More non-COVID-19

samples could provide valuable information to optimize the cutoff value.

Both the Abbott SARS-CoV-2 IgG and the Roche Anti-SARS-CoV-2 assays list EDTA plasma and heparin plasma as acceptable specimens. In this study, 139 pairs of EDTA plasma and heparin plasma specimens showed 100% concordant results when tested by both assays.

The linear regression of the COI index showed minimal bias for the Roche assay between EDTA and heparin plasma (Fig. 2) . In contrast, the Abbott assay generated about 17%

higher signal in average in heparin plasma compared with that in EDTA plasma at high S/C indices (>1.4). The bias was less than 10% when S/C index was below the cutoff value of 1.4. However, the qualitative results of heparin and EDTA plasma remained the same.

Both the Abbott (y) and the Roche (x) are qualitative assays based on quantitative measurement. There is a positive correlation (y = 0.1199x + 1.3349, R=0.772) between the relative signals of the antibodies generated by the Abbott and the Roche assays in COVID-19 patients (Fig. 1) . For both the Roche and Abbott assays, there was no significant difference among the signal index of the specimens collected at 0-6 days, 7-13 and ≥14

days post RT-PCR confirmation by Mann-Whitney U test when analyzed as a whole (Fig.  3 ). However, we did observe longitudinal increases in signal index in most patients, when analyzed individually (Fig. 4) . This may be due to two reasons. First, the patients visited hospital for healthcare at different days after exposure to the virus. Second, there is a large inter-individual variation in patients' antibody response. This implies that a single quantitative signal index is of little value in estimating the stage of the disease.

The Roche assay is designed to detect the total antibodies and the Abbott assay is to detect IgG antibody. In theory, the Roche assay should better detect early seroconversion.

This advantage was true for 11 patients in this study, but the other 2 patients showed early seroconversion by the Abbott assay. It is unknown whether this finding is a consequence of differing antigen epitopes used in the immunoassay although both assays utilize the nucleocapsid protein as target protein. This may also reflect the very heterogeneous nature of patients' immune response to SARS-CoV-2. One study showed that patients developed IgG first, or IgM first or both IgG and IgM antibodies simultaneously (9) . Another study showed that patients developed IgM and IgA within a median of 5 days, and IgG with a median of 14 days after symptom onset (8) . Similar to the finding in another report (14), we observed a few patients had transient decrease in the signal index of the Roche assay in the late stage, which may represent the switching from IgM to IgG production as well as different affinities between IgG and IgM (Fig. 4) .

More studies are needed to understand the kinetics of the antibody development in this novel disease.

In summary, this study demonstrates that the analytical performance of both the Abbott SARS-CoV-2 IgG and the Roche Anti-SARS-CoV-2 immunoassays is consistent with their claims. The clinical performance between these two immunoassays are similar, but the Roche assay may be slightly more sensitive for patients tested within 0-6 days after first positive RT-PCR of SARS-CoV-2. 

A novel coronavirus from patients with pneumonia in china

Clinical features of patients infected with 2019 novel coronavirus in wuhan, china

Laboratory diagnosis and monitoring the viral shedding of 2019-ncov infections

Antibody responses to sars-cov-2 in patients of novel coronavirus disease 2019

Viral load dynamics and disease severity in patients infected with sars-cov-2 in zhejiang province, china

Interpreting diagnostic tests for sars-cov-2

Failing the test -the tragic data gap undermining the u.S. Pandemic response

Profiling early humoral response to diagnose novel coronavirus disease (covid-19)

Antibody responses to sars-cov-2 in patients with covid-19

Virological assessment of hospitalized patients with covid-2019

Comparison of four new commercial serologic assays for determination of sars-cov-2 igg

Two-sided confidence intervals for the single proportion: Comparison of seven methods

Severe acute respiratory syndrome coronavirus 2-specific antibody responses in coronavirus disease 2019 patients

Clinical performance of the Roche SARS-CoV-2 serologic assay

Clinical performance of the Elecsys electrochemiluminescent immunoassay for the detection of sars-cov-2 total antibodies

Clinical performance of two SARS-CoV-2 serologic assays

Performance characteristics of the abbott architect sars-cov-2 igg assay and seroprevalence in boise, idaho

100% (16/16) (95% CI