key: cord-255290-p64apuk1
authors: Matheeussen, Veerle; Corman, Victor M; Donoso Mantke, Oliver; McCulloch, Elaine; Lammens, Christine; Goossens, Herman; Niemeyer, Daniela; Wallace, Paul S; Klapper, Paul; Niesters, Hubert GM; Drosten, Christian; Ieven, Margareta
title: International external quality assessment for SARS-CoV-2 molecular detection and survey on clinical laboratory preparedness during the COVID-19 pandemic, April/May 2020
date: 2020-07-09
journal: Euro Surveill
DOI: 10.2807/1560-7917.es.2020.25.27.2001223
sha: 
doc_id: 255290
cord_uid: p64apuk1

Laboratory preparedness with quality-assured diagnostic assays is essential for controlling the current coronavirus disease (COVID-19) outbreak. We conducted an external quality assessment study with inactivated severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) samples to support clinical laboratories with a proficiency testing option for molecular assays. To analyse SARS-CoV-2 testing performance, we used an online questionnaire developed for the European Union project RECOVER to assess molecular testing capacities in clinical diagnostic laboratories.

Laboratory preparedness with quality-assured diagnostic assays is essential for controlling the current coronavirus disease (COVID-19) outbreak. We conducted an external quality assessment study with inactivated severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) samples to support clinical laboratories with a proficiency testing option for molecular assays. To analyse SARS-CoV-2 testing performance, we used an online questionnaire developed for the European Union project RECOVER to assess molecular testing capacities in clinical diagnostic laboratories.

Extensive laboratory testing is a prerequisite in containing and mitigating the impact of the ongoing coronavirus disease (COVID-19) pandemic while neither vaccine nor treatment are available [1] . Access to reliable diagnostic assays and adequate testing capacity for the causing severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is essential for preparedness and response strategies worldwide [2] . Molecular methods such as RT-PCR are fundamental to early SARS-CoV-2 detection in suspected cases. Many inhouse and commercial assays have been developed rapidly [3, 4] . However, the quality and diagnostic performance of these tests have not been adequately validated; the World Health Organization (WHO) has therefore encouraged laboratories to participate in external quality assessment (EQA) schemes for this novel virus [5] . Here, we present the results of a first EQA on molecular detection of SARS-CoV-2 introduced by Quality Control for Molecular Diagnostics (QCMD, Glasgow, Scotland), an independent International EQA organisation. We also present data from a survey by the European Union (EU) project RECOVER, a COVID-19-related project originating from the EU-initiative Platform for European Preparedness Against (Re-) emerging Epidemics (PREPARE), assessing the molecular testing capacity and throughput of clinical laboratories in 36 countries.

Between 6 April and 20 May 2020, each participating laboratory received a blinded panel of eight samples, with five samples containing serial 10-fold dilutions (2.30-5.30 dPCR log10 RNA copies/mL, one duplicated) of non-infectious SARS-CoV-2-positive supernatant obtained from Vero cell culture, two samples with cell culture-derived common human coronavirus (HCoV-NL63, HCoV-OC43) and one negative control with transport medium only. Supernatants were clarified by centrifugation at 300 × G for 10 min. In case of SARS-CoV-2, the material was heated (65 °C, 4 h) and gamma-irradiated (30 kGy) for inactivation. The 1.0-mL samples were pretested using the E gene assay (modified from [6] ), and quantified by droplet digital PCR (Bio-Rad, Hercules, United States (US)). The EQA panels were shipped on dry ice. Participants were asked to treat and test the material like clinical samples using their routine molecular assay workflows and to report their results together with workflow details through a dedicated online reporting system. Laboratories submitted one dataset per applied workflow with extraction and amplification method used to test the provided samples.

In this first EQA round, 365 of 406 registered laboratories from 36 countries (25 European, nine Australian-Asian and two North American) submitted a total of 521 datasets with qualitative results. All participants received a rapid report after submission of their results, followed by an EQA report with individual performance and peer group assessment on completion of the study. The panel composition and overall performance per sample are shown in Table 1 .

Detecting all SARS-CoV-2-positive samples and the negative sample correctly was defined as an acceptable level of proficiency. All core samples were correctly reported by 86.3% (315/365) of participating laboratories and in 83.1% (433/521) of datasets. When including the results reported for the two educational specificity samples, the overall percentage of correct results was 84.7% (309/365) on a laboratory level and 81.8% (426/521) for datasets. Incorrect results (including false-positive or not determined as laboratories should be able to report clear results by molecular methods) were reported for the true negative sample CVOP20S-05 in 14 (2.7%) datasets (with three falsepositive and 11 not determined), for the SARS-CoV-2 negative educational sample CVOP20S-02 in 16 (3.1%) datasets (five false-positive, 11 not determined), and for the other SARS-CoV-2 negative educational sample CVOP20S-04 in 15 (2.9%) datasets (three falsepositive, 12 not determined). At the laboratory level, 10 laboratories showed specificity issues by reporting false-positive or indeterminate results for the negative control and/or the two educational specificity samples, while 41 laboratories did not detect one or both of the two low-concentration samples (CVOP20S-08 and -03). Three laboratories showed both sensitivity and specificity issues and another two did not reproduce the of Virology, Berlin, Germany; human coronaviruses HCoV-NL63 and HCoV-OC43 cultivated by the University Medical Center Groningen, Groningen, the Netherlands, and sample specifications selected based on past performance data from regular provided coronavirus EQA schemes; negative control sample contained transport medium only (which was used as sample matrix for the EQA samples). b Values obtained using a dPCR assay (modified from [6] ). Samples CVOP20S-07, -01, -03 and -08 are in a calibrated dilution series. Logistic regression statistics, performed in SAS version 9.4 (SAS Institute Inc., Cary, US), was used to assess whether common technical factors, such as the extraction and amplification groups (by methods) listed in Table 2 , influenced participant performance. Both extraction and amplification method were statistically significantly associated with correct classification of all samples (p < 0.001 for both). While the overall percentage of correct results was above 90% for each analysed group, certain groups showed statistically significant differences compared with the corresponding reference level, namely 'any other extraction method' or 'any real-time in-house PCR'. As an example, for one fully automated approach (Certest VIASURE SARS-CoV-2 S gene real-time PCR kit (Certest, Zaragoza, Spain) used with the BD MAX system (Becton, Dickinson and Company, Franklin Lakes, US)) combining RNA extraction and subsequent real-time PCR (with only 90.3% and 90.8% overall correct results, respectively), all 14 false-negative results reported by 13 laboratories could be clearly linked to sensitivity issues with the two lowconcentration samples in the panel. However, some of the observed differences for the groups listed in Table  2 could also be due to improper handling of the assays and/or the samples, since laboratories using the same assays showed large variations in their reported qualitative results (e.g. because of differences in the parameters used for indeterminate results).

The data from this EQA study provide a comparison of available molecular assays for SARS-CoV-2 detection. About 23% of the used assays were in-house assays which performed at least as well as commercial ones or even better. The overall qualitative performance of the participating laboratories was at an acceptable level and showed similar rates of successful participation as compared with the molecular EQA for SARS (provided by WHO/European Network for Diagnostics of "Imported" Viral Diseases (ENIVD)) during the outbreak in 2002 and 2003, with similar requirements and rapid developments for diagnostics, where 51 of 58 laboratories from 38 countries achieved an acceptable level of proficiency [7] . Our EQA pilot study presented here revealed that analytical sensitivity and specificity remained variable.

Procedures should be reviewed where false-positive or indeterminate results have been reported which may lead to misdiagnosis and affect clinical decision making. As highly sensitive methods are required for early COVID-19 diagnostic screening, two low-concentration samples were included in the EQA panel close to the detection limit of published or commercial assays [3, 4, [8] [9] [10] . Laboratories that were unable to detect low-concentration samples, or whose methods showed Cq values greatly different from the provided medians, should strive to improve the sensitivity of their molecular assays to prevent false-negative results in respiratory samples with low viral concentrations from SARS-CoV-2-infected patients, e.g. during the early phase of infection.

In parallel with the EQA study, laboratories were surveyed to assess their challenges in implementing and executing molecular testing capacity and throughput for SARS-CoV-2 detection, also in April and May 2020.

Of the 365 laboratories that participated in our EQA pilot study, five were excluded from the survey as they were engaged in assay development rather than in routine SARS-CoV-2 detection.

Almost 80% of the participating laboratories (n = 360) are capable of generating a SARS-CoV-2 PCR result within 24 h after receiving the sample (Supplementary Figure S1 ). This turnaround time is comparable to the results of a smaller survey (n = 87 laboratories) performed earlier in the COVID-19 pandemic [11] . The daily test capacity, however, was much higher, with 48% of the laboratories capable of analysing more than 250 samples per day, while only 41% in the previous survey could handle more than 50 samples [11] . More than 70% of the laboratories indicated that collaboration with reference laboratories and (inter)national public health institutes has helped in establishing SARS-CoV-2 testing capability and capacity. However, challenges in implementation and execution of the testing remain, with lack of personnel/time and equipment being the predominant ones (Supplementary Figure S2) . Compared with the earlier survey, challenges did shift over time, with lack of funding, an authorised mandate or unavailability of positive control material and/or a specificity panel becoming less important barriers.

Molecular testing capacity and throughput on clinical diagnostic level have been rapidly implemented and should be supported by proficiency testing panels. EQA schemes, such as the one presented here, are a good opportunity for laboratories to assess the performance of their assays against international peer groups in line with agreed clinical practice, based on well-characterised samples, to identify any weaknesses with their procedures or methods. Laboratories should be aware of the limitations of their assays and perform their own validation and verification in line with ISO 15189 or equivalent requirements. Also, EQA data can provide valuable information for post-market surveillance of commercial assays, which is of particular importance in outbreak situations where data about the relative performance of methods are still limited and further clinical evaluations are ongoing. This way, the quality of COVID-19 diagnostic testing can be continuously ensured. Regional-specific EQA studies can also be done, but require a different approach in order to ensure comparability. For 2020, two further EQA programmes are planned by QCMD: a follow-up SARS-CoV-2 EQA study and a respiratory EQA scheme comprising SARS-CoV-2 and other respiratory pathogens [12] .

Laboratory support for COVID-19 in the EU/EEA. Stockholm: ECDC

2019 novel coronavirus (2019-nCoV): strategic preparedness and response plan. Geneva: WHO

Molecular assays to diagnose COVID-19: Summary table of available protocols. Geneva: WHO

COVID-19 diagnostics resource centre. Geneva: FIND

Laboratory testing for 2019 novel coronavirus (2019-nCoV) in suspected human cases. Geneva: WHO

Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR

SARS molecular detection external quality assurance

Coronavirus disease 2019 (COVID-19) EUA information

Limits of detection of 6 approved RT-PCR kits for the novel SARScoronavirus-2 (SARS-CoV-2)

Evaluation of a quantitative RT-PCR assay for the detection of the emerging coronavirus SARS-CoV-2 using a high throughput system

Quality Control for Molecular Diagnostics (QCMD). Respiratory special. Glasgow: QCMD; 2020

This work forms part of the Rapid European COVID-19 Emergency Research response (RECOVER) project, and collaborating networks PREPARE ( www.prepare-europe.eu ), COMBACTE LAB-Net ( www.combacte.com/about/lab-net/ ), EVD-LabNet ( www.evd-labnet.eu ) and EVAg ( www.european-virus-archive.com ).We are greatly indebted to all our colleagues who participated in the QCMD molecular EQA pilot study and RECOVER survey on clinical laboratory preparedness. Special thanks to Alastair Ricketts and his team (Qnostics Ltd., Glasgow, Scotland) for contributing to the material assessment. We further thank Harry Staines (Sigma Statistical Services, Balmullo, UK) for supporting and reviewing the statistical analyses, and Matthias Niedrig (coordinator from the former ENIVD network) for critically reading the manuscript. RECOVER is funded by the European Union's Horizon 2020 research and innovation programme under grant agreement No 101003589.

This is an open-access article distributed under the terms of the Creative Commons Attribution (CC BY 4.0) Licence. You may share and adapt the material, but must give appropriate credit to the source, provide a link to the licence and indicate if changes were made.Any supplementary material referenced in the article can be found in the online version.