key: cord-325910-qiay8n43 authors: Green, D. A.; Zucker, J. E.; Westblade, L. F.; Whittier, S.; Rennert, H.; Velu, P.; Craney, A.; Cushing, M.; Liu, D.; Sobieszczyk, M. E.; Boehme, A. K.; Sepulveda, J. title: Clinical Performance of SARS-CoV-2 Molecular Testing date: 2020-05-08 journal: nan DOI: 10.1101/2020.05.06.20093575 sha: doc_id: 325910 cord_uid: qiay8n43 Molecular testing for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is the gold standard for diagnosis of coronavirus disease 2019 (COVID-19), but the test clinical performance is poorly understood. From 3/10/2020-5/1/2020 NewYork-Presbyterian laboratories performed 27,377 SARS-CoV-2 molecular assays from 22,338 patients. Repeat testing was performed in 3,432 patients, of which 2,413 had negative and 1,019 had positive first day results. Repeat-tested patients were more likely to be older, male, African-American or Hispanic, and to have severe disease. Among the patients with initially negative results, 18.6% became positive upon repeat-testing. Only 58.1% of any-time positive patients had a result of "detected" on the first test. The clinical sensitivity of COVID-19 molecular assays is estimated between 66.2% and 95.6%, depending on the unknown number of false negative results in single-tested patients. Conversion to a negative result is unlikely to occur before 15 to 20 days after initial testing or 20-30 days after the onset of symptoms, with 50% conversion occurring at 28 days after initial testing. Forty-nine initially-positive patients converted to negative and then back to positive in subsequent days. Conversion from first day negative to positive results increased linearly with each day of testing, reaching 25% probability in 20 days. In summary, our study provides estimates of the clinical performance of SARS-CoV-2 molecular assays and suggests time frames for appropriate repeat testing, namely 15 to 20 days after a positive test and the same or next 2 days after a negative test in a patient with high suspicion for COVID-19. ). Our data show higher age (median = 59.9 vs. 53.4, 147 p<0.001), higher frequency of male gender (52.2% vs. 44.3%, p<0.001), and different 148 distribution of self-reported race and ethnicity in repeat-tested as compared to single-149 tested patients, with African-Americans and Hispanics / Latinos being more likely to be 150 repeat-tested (p<0.001). At the time of the first test order, admitted patients were 151 significantly more represented in the repeat-tested population in contrast to patients 152 visiting the emergency department and outpatients (p<0.001). Compared with single-153 tested patients, repeat-tested patients were 3.2 times more likely to be admitted to the ICU 154 (29.7% vs. 9.4%, p<0.001), 4 times more likely to be intubated (24.4% vs. 6.1%, p<0.001), 155 3.4 times more likely to decompensate (27.6% vs. 8.1%, p<0.001) and 1.7 times more likely 156 to die during the observation period (8% vs. 4.7%, p=0.038). 157 The characteristics of the SARS-CoV-2 tests performed in repeat-tested compared to single-159 tested patients are described in Supplementary Table 4 . The vast majority of tests were 160 performed with the cobas 6800 and the use of the various assays was not significantly 161 different between repeat and single-tested patients, except for a small number of patients 162 initially tested with the Thermo-Fisher 7500 assay, which was more likely in repeat-tested 163 patients (p<0.001, standardized Pearson residuals for repeat-tested patients = 3.9). 164 Among all the repeat-tested patients, 23.2% were positive on the first test (26.5% when 165 indeterminate results were included). If a negative test was repeated on the first day, the 166 positivity rate increased to 26.4% (29.7% with indeterminate results). Overall positivity 167 among repeat-tested patients over the course of the study period was 39.9% in contrast to 168 49% for single-tested patients (p<0.001). When indeterminate results were counted as 169 positive, 42.9% of repeat-tested patients were positive over the course of the study period 170 in contrast to 50% of single-tested patients (p<0.001). 171 Indeterminate results are generally considered presumptive positive and occur when only 172 one of two molecular targets is detected. In our repeat-tested patients with an initial result 173 of "Indeterminate", 54.4% ultimately had a result of "Detected", as compared to 7% that 174 remained indeterminate upon repeat testing and 38.6% that converted to a negative status 175 during our study period, thus suggesting that it is acceptable to consider these patients 176 positive. However, it is unclear whether the initially indeterminate patients that converted 177 to negative were false positives or presented with low viral loads. 178 In contrast, a result of "Invalid" reflects the failure to amplify the built-in control and is 179 likely related to poor sampling or inadequate RNA extraction usually due to high viscosity 180 of the samples. In our study of repeat test patients, 52.7% ultimately became positive, 1.2% 181 repeated as "Indeterminate" and 45.6% repeated as 'Not Detected'. For the analysis of 182 clinical sensitivity of SARS-CoV-2 molecular tests, we considered patients with invalid 183 results as "negative", as these results can be considered clinically false negatives in the 184 sense that the patient may be infected and the test failed to yield a positive result. In 185 practice, invalid results should always be repeated, preferably with a new sample, as the 186 results are unpredictable. 187 Initial negative, invalid, or indeterminate SARS-CoV-2 test results were much more 188 frequent among repeat-tested patients (Supplementary Table 4 , p<0.001) and conversely, 189 patients without a positive initial result were more likely to be repeated (21%) than 190 initially positive patients (8%, p<0.001). 191 A subset of the cobas 6800 tests (N=5,343) had cycle threshold (Ct) values available for 192 analysis. The Ct represents the PCR cycle, interpolated to two decimal digits, at which the 193 real-time fluorescent signal crosses a pre-defined threshold for positivity. The Ct is 194 inversely proportional to the concentration of viral RNA. The cobas SARS-CoV-2 RT-PCR 195 assay amplifies two specific targets: Target 1 is located in the ORF1ab non-structural 196 region that is unique to SARS-CoV-2 and Target 2 is a conserved region of the structural 197 protein envelope E-gene common to all members of the Sarbecovirus sub-genus of 198 coronavirus, which include SARS-CoV-2 and SARS-CoV (8, 9) . The cobas assay also includes 199 an internal control for assay performance that has no homology to the coronaviruses. In this study, we classified repeat-tested patients in two groups according to their initial 210 SARS-CoV-2 results: those that were initially positive, for whom repeat testing was most 211 likely intended to ascertain recovery and non-infectiousness, and those that had an initial 212 result of negative, indeterminate, or invalid, in whom persistent clinical suspicion for 213 COVID-19 likely motivated repeat ordering of the test. Supplementary Table 4 shows the 214 different clinical characteristics between repeat-tested and single-tested patients. 215 For the time-dependent analysis of conversion rates, we considered "initially positive" 216 patients with any "Detected" or "Indeterminate" SARS-CoV-2 result obtained during the 217 first calendar day of testing rather than the first positive test, to reduce bias due to 218 nasopharyngeal sampling inadequacy (Table 1) . Conversely, patients without a result of 219 "Detected" or 'Indeterminate" in the first day were labeled as "initially negative". Among 220 the 2,413 initially negative repeat-tested patients, 18.6% became positive upon repeat 221 testing on subsequent days (Supplementary Table 5 Table 6 ). Among the patients with repeat testing who had initial results of 227 "Invalid" (241), "Not Detected" (2280), or "Indeterminate" (114), 4.2% had 'Detected' 228 results upon repeat testing in the first day of testing (Supplementary Table 5 ). This 229 increase in positivity rate most likely results from a false-negative initial test due to pre-230 analytic factors such as sample inadequacy, incorrect swabbing technique, or stochastic 231 sampling bias from low viral loads in the patient nasopharynx. After the first day of testing, 232 repeating invalid, negative, or indeterminate results on the same day resulted in about 233 0.8% of additional positive results per day, for a total of 18.4% positives that were missed 234 by the first test. 235 Among the 1,371 repeat-tested patients with one or more SARS-CoV-2 results of 236 "Detected", which can be assumed to be truly infected, only 58.1% were resulted as 237 "Detected" on the initial test (Supplementary Table 5) , and only 66.2% were reported as 238 "Detected" on the first day (Table 1) . Considering 'Detected' and "Indeterminate" as 239 positive, 1,471 repeat-tested patients had one or more SARS-CoV-2 positive results over 240 time; only 61.9% were positive on the initial test and only 69.3% had a positive result on 241 the first day (Table 1) . These data provide an estimate of the clinical sensitivity of the assay 242 in the repeat-tested population, and establish a baseline to look at conversion rates from 243 negative to positive. 244 Table 2A shows the number of patients who had an initial result of "Not Detected" on day 1 245 who converted to a SARS-CoV-2 positive status grouped per time after the initial test. 246 Conversely, Table 2B shows rates of conversion to negative for patients with a status of 247 "Detected" on day 1. Supplementary Figure 1 shows the distribution per day after onset of 248 symptoms (a and b) or after initial testing (c to f) of conversions from positive to negative 249 (left-side) and from negative to positive (right-side) in the two groups of repeat-tested 250 patients. For this analysis, positive status included "Detected" and "Indeterminate" and 251 negative status included results of "Not Detected" with "Invalid" results excluded. Among 252 the initially positive patients, the unadjusted distributions show a peak of conversion to 253 negative between 30 and 40 days after symptoms or around 20 days after initial testing. 254 Less than 10% of the patients who converted to negative converted before 15 days after 255 onset of symptoms or 10 days after initial testing. In contrast, among the patients with 256 initially negative results who converted to positive, most conversions occurred 10 days or 257 less after onset of symptoms, and in the first 1-3 days after initial testing. 258 Since we cannot be certain about the conversion rates due to a significant proportion of 259 3. Conversely, an event is defined as a conversion from a "Not Detected", or 268 "Invalid" result at day 1 to "Detected" or "Indeterminate" on subsequent days. 269 4. If the last test result was unchanged relative to the first day result, the patient 270 was considered right-censored at that time. 271 5. Only the first day and either the censoring day or the event day were used for 272 each patient and indeterminate results were ignored. 273 The results show that the probability of converting from positive to negative in the initially 274 positive, repeat-tested population is minimal until about 15 to 20 days after initial testing 275 and reaches 50% at 28 days (95% CI = 27 to 29 days, Figure 2 ). In the initially negative 276 repeat-tested population, the risk linearly increases every day with a 25% probability of 277 conversion to positive of 20 days (95% CI = 16 to 23 days, Figure 3 ). 278 Since lack of conversion could be due to death of the patient, which occurred in about 8% 279 of the repeat-tested population, we performed a competing risk analysis with death as the 280 alternative event, using the timereg R package (10), and did not find significant differences 281 in either the positive to negative or the negative to positive Kaplan-Meier cumulative 282 probability of conversion (results not shown). 283 Interestingly, among the repeat-tested patients we have identified 11 patients that 284 converted from an initial SARS-CoV-2 negative result to positive and back to negative over 285 several days. We also identified 49 initially positive patients that became negative and later was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Our results from repeat-tested patients can be used to estimate the clinical sensitivity of 291 the SARS-CoV-2 molecular testing in the population of patients that were selected for 292 repeat testing, in contrast to the general population of tested patients. With all likelihood, 293 most repeat testing on initial negative patients was performed either to follow a history of 294 exposure, when the clinical profile did not fit the initial results, or when clinical 295 presentation deteriorated after the initial result. Consistent with this hypothesis, repeat-296 tested patients were more likely to be older, male and of non-Caucasian race than single 297 tested patients (Supplementary Table 3 Table 2) . Importantly, repeat-tested patients had worse 299 outcomes as demonstrated by higher rates of decompensation, intubation, and mortality. 300 Interestingly, repeat-tested patient who converted from positive to negative tended to be 301 younger and present as outpatients, as compared to patients that remained positive, but 302 there were no significant differences in clinical outcomes (Supplementary Table 7 ). In 303 contrast, repeat-tested patients that remained negative tended to be inpatients, have 304 longer admission duration, and were more likely to be extubated than those who converted 305 to positive (Supplementary Table 8 ), suggesting that a significant number of repeat-testing 306 in SARS-CoV-2 negative patients was performed in inpatients admitted before the 307 pandemic or for non-COVID-19 reasons. Indeed, repeat-tested patients admitted before 308 March 1st, 2020 were much more likely to have a persistent negative result (71.4%) than 309 those admitted after March 1st, 2020 (39.7%, p<0.001). 310 All rights reserved. No reuse allowed without permission. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 8, 2020. . In the absence of a more sensitive gold-standard, the repeat-tested patients with an 311 eventual positive result can be considered true positives, as the analytical specificity of 312 molecular testing is very high (3, 8, 9, 11, 12) . It may be tempting to add all the 9,272 313 single-tested positive patients to the 1,371 repeat-tested positive patients to determine 314 clinical sensitivity. However, we don't know how many of the "Not detected" single-tested 315 patients are false negatives, especially given the high frequency of asymptomatic or mildly 316 symptomatic COVID-19 patients (13-15). Therefore, considering only the positive patients was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 8, 2020. . https://doi.org/10.1101/2020.05.06.20093575 doi: medRxiv preprint 2. A significant number of samples are improperly collected, and repeat testing 335 increases the probability of detection; this is particularly likely in the first day of 336 testing, when suspicion may be high but the initial test results were negative or 337 inconclusive, as shown in Supplementary Table 5 . was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 8, 2020. . https://doi.org/10.1101/2020.05.06.20093575 doi: medRxiv preprint with COVID-19. Rather, these data suggest a pattern of repeated ordering in uninfected 359 inpatients with a lower likelihood of conversion. 360 Our analysis of repeat-tested patients with an initial positive result, using the Kaplan-Meier 361 estimator, indicates that conversion to a negative result is unlikely to occur until about 15 362 to 20 days after initial testing or 20 to 30 days after start of symptoms, when the odds ratio 363 significantly increase ( likely reflects sampling inadequacy, often due to excess mucous in the sample. Our finding 370 of significantly higher Ct in repeat-tested patients as compared to single-tested patients 371 supports this hypothesis. 372 This is an observational study without selection bias for the laboratory data, as all results 374 were included in the analysis. However, the clinical data was restricted to a subset of 375 patients seen at Columbia University Irving Medical Center campuses. Nevertheless, rates 376 of positivity and demographic variables captured in the laboratory dataset were not 377 significantly different between the other campuses, suggesting that the population in the 378 clinical dataset is generally representative of the New York City patients tested for COVID-379 19. Other limitations of the study of repeat-tested patients include: 1) only 15% of the total 380 patient's tested had repeat testing done and 2) ordering of repeat testing was at the 381 discretion of the health care providers and not performed according to a standard protocol, 382 All rights reserved. No reuse allowed without permission. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Funding: This research received no specific grant from any funding agency in the public, 403 commercial, or not-for-profit sectors. 404 All rights reserved. No reuse allowed without permission. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 8, 2020. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 8, 2020. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 8, 2020. . was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 8, 2020. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 8, 2020. , c, e, and g) and positive (b, d, f, and h) results. For the analyses in a to d only the first conversion events (i.e. negative to positivea and c -or positive to negativeb and d) were considered. N represents the total numbers of patients analyzed. The distributions in a to d are represented by density-adjusted histograms (in grey bars), kernel probability density lines (in blue), unadjusted cumulative distributions (red line) and Kaplan-Meier hazard rates (green); the timing of individual results are represented by the black marks along the x-axis. For the Kaplan Meier estimated hazard rates (e to h), in addition to the conversion events, patients were considered censored at the time of the last unchanged result were included and the number of patients at risk for conversion at each time point are shown in g and h. All rights reserved. No reuse allowed without permission. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 8, 2020. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 8, 2020. . https://doi.org/10.1101/2020.05.06.20093575 doi: medRxiv preprint All rights reserved. No reuse allowed without permission. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Mean (sd) 6.9 (13.7) 6.9 (17.0) 6.9 (16.9) Q1, Q3 2.5, 12.5 2.7, 7.9 2.7, 7.9 All rights reserved. No reuse allowed without permission. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 8, 2020. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 8, 2020. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 8, 2020. . All rights reserved. No reuse allowed without permission. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 8, 2020. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 8, 2020. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 8, 2020. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 8, 2020. Pearson's Chi-squared test (adjusted for multiple comparisons) Linear Model ANOVA Trend test for ordinal variables (adjusted for multiple comparisons) Pearson's Chi-squared test (adjusted for multiple comparisons) Linear Model ANOVA Trend test for ordinal variables (adjusted for multiple comparisons)