key: cord-349545-w7c2tu5a authors: Wang, Mengmeng; Chen, Dayang; Wu, Wei; Tang, Huamei; Kan, Lijuan; Zong, Zengyan; Dou, Xiaowen; Ji, Xiang; Xiong, Dan; Zhang, Xiuming title: Analytical performance evaluation of five RT‐PCR kits for severe acute respiratory syndrome coronavirus 2 date: 2020-10-27 journal: J Clin Lab Anal DOI: 10.1002/jcla.23643 sha: doc_id: 349545 cord_uid: w7c2tu5a BACKGROUND: We aimed to evaluate the analytical performance of five commercial RT‐PCR kits (Genekey, Daan, BioGerm, Liferiver, and Yaneng) commonly used in China, since such comparison data are lacking. METHODS: A total of 20 COVID‐19 confirmed patients and 30 negative nasopharyngeal swab specimens were analyzed by five kits. The detection ability of five RT‐PCR kits was evaluated with 5 concentration gradients diluted by a single positive sample. The limit of detection was evaluated by N gene fragment solid standard. Two positive clinical specimens were used to evaluate the repeatability and imprecision. Finally, we used six human coronaviruses plasmid and four respiratory pathogens plasmid to check for cross‐reactivity. RESULTS: The positive detection rate was 100% for Genekey, Daan, and BioGerm,and 90% for Liferiver and Yaneng in 20 clinical SARS‐CoV‐2 infection. The coincidence rate of five kits in 10 negative samples was 100%. The detection rate of target genes for Daan, BioGerm, Liferiver, and Yaneng was 100% from Level 1 to Level 3. In Level 4, only Daan detection rate was 100%. In Level 5, five kits presented poor positive rate. The limit of detection declared by each manufacturer was verified. The repeatability for target genes was less than 5% and so did the total imprecision. There is no cross‐reactivity of five kits with six human coronaviruses and four respiratory pathogens for ORF1ab and N gene. CONCLUSIONS: Five RT‐PCR kits assessed in this study showed acceptable analytical performance characteristics and are useful tools for the routine diagnosis of SARS‐CoV‐2. Coronavirus disease 2019 , caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), had caused more than 5 million cases confirmed infections and 337 736 reported deaths until May 24, 2020, as reported by World Health Organization. The epidemic has spread to over 180 countries worldwide, and case numbers predicted to rise in the coming months. 1, 2 To facilitate identification of infected people and ensure appropri- Some suspected patients exhibited typical clinical pneumonia symptoms, such as fever, cough, myalgia, fatigue, or image characteristics, but were negative in RT-PCR testing in clinical practice. 4, 5 This raises the question of whether there are differences in detection performance among RT-PCR testing kits. 6, 7 RT-PCR testing kits diagnostic accuracy is depending on many factors, such as skilled laboratory staff, sample types and collection, transportation conditions, the test kit quality, and so on. Wang et al reported that oropharyngeal swabs showed lower positive rate than nasopharyngeal swabs. 8 Pan et al found that thermal inactivation adversely affected the efficiency of RT-PCR for SARS-CoV-2 detection samples with low viral loads. 9 Wang et al compared the limit of detection (LoD) of five RT-PCR kits using real viral RNA. 7 However, there is still a lack of performance evaluation of RT-PCR kits from different manufacturers under the strict quality evaluation system. In the work, we presented the analytical performance evaluations of five RT-PCR kits using nasopharyngeal swabs samples from patients with confirmed SARS-CoV-2 infection, and negative nasopharyngeal swabs samples ( Figure 1 ). Our work may promote implementation and standardization across public health laboratories globally and be helpful for immediate decisions about clinical care and preventive measures. The five commercial RT-PCR kits were selected for the study (Table 1) This study only focused on the corresponding Lot number reagent. To use the patient's nasopharyngeal swab as nucleic acid detection sample, all samples were dealt with nucleic acid extraction and purification kit (Health Gene Technologies Co. Ltd) on a Smart LabAssist-32 platform (Taiwan Advanced Nanotech Inc). All operation steps followed manufacturer recommended protocols. The reaction procedure of ABI7500 was set according to the reagents specifications of different manufacturers, and the test results were judged as negative or positive. During each run, both positive and negative controls were included to ensure that proper PCR responses are not subjected to carryover. To compare the detection capability for clinical specimens, a total of 30 specimens were separated for investigation using five RT-PCR kits, all of which are based on TaqMan technology. Daan, BioGerm, and Genekey target ORF1ab and N gene. Liferiver and Yaneng target ORF1ab/N/E and ORF1ab/N/S gene, respectively. The positive and negative results were interpreted based on the manufacturer's instructions. To further confirm the detection ability of the five kits, a positive clinical specimen (Ct: ORF1ab 26.99, N: 28.19) was diluted with 5-fold using RNase-free water, and the resulting dilution is considered as Level 1. A total of 5 concentration levels, Level 1, Level 2(1/3 Level 1), Level 3(1/9 Level 1), Level 4(1/27 Level 1), and Level 5 (1/81 Level 1), were tested with 5 replicates per concentration with the five kits. Each target gene was tested based on cycle threshold (Ct) values and amplification curves obtained during RT-PCR. Then, the detection rate (number of positive results/total number of measurements) was used to evaluate the detection ability of the five kits. The limit of detection (LoD) is the lowest concentration level that can be detected with specified kit. The N gene fragment solid standard (GW-CRPM001, GeneWell, China) was diluted into LoD that manufacturer claimed of each kit (500 copies/mL for Daan and 1000 copies/mL for others) using RNase-free water. Twenty replicates at LoD were prepared following the sample preparation procedure given in each kit. A detection rate of ≥95% for positive results means that the results conform to the LoD that the manufacturers claimed. Two positive clinical specimens (P1: low positive and P2: moderate positive) were used to evaluate the repeatability and imprecision. According to modified EP15-A protocol, each sample was tested five times a day for five days. Coefficient of variation (CV) and standard deviation (SD) were calculated separately for each kit. The statistical software R version 3.6.1 (http://www.r-proje ct.org) was used for data evaluation. Excel 2010 and GraphPad Prism5.0 (GraphPad Software, Inc) were used for statistical analysis. The However, for the other two target genes, the positive rate of Liferiver The detection results of five concentrations were shown in Table 2 . The results showed that the positive coincidence rate was 100% The results of five kits for repeatability and imprecision evaluation were shown in (P2), respectively. The total imprecision for ORF1ab gene was from 0.14% (P1) to 1.1% (P2) and N gene was from 0.16% (P1) to 0.52% (P2). Genekey has the best imprecision for ORF1ab gene with a CV value 0.24% in P1 and 0.49% in P2. BioGerm has the best repeatability for ORF1ab gene in both P1 (0.11%) and P2 (0.45%). Five kits have superior repeatability and imprecision for N gene in P1 and P2 expect Liferiver which has poorest repeatability and imprecision. It worth note that Genekey has a poorer imprecision for N gene in P2 (0.56%), rank only second to Liferiver. However, It has the best imprecision for N gene of all kits in P1. The repeatability was all less than 5%, and imprecision was less than 10% for all the five kits. The repeatability and imprecision of ORF1ab and N genes in P2(low positive sample) were generally higher than that in P1(strong positive sample). Six human coronaviruses and four respiratory pathogens were selected as interfering substance to evaluate the cross-reactivity. No positive result was obtained in testing of 30 negative clinical samples by using five kits for ORF1ab and N gene. Rapid detection of infectious virus will be useful for outbreak in- Positive results should not be reported according to Liferiver kit instructions. Another study showed that the E gene has a 20% (2/20) missed detection rate for the kit. 10 In addition, we used 1 low positive sample and 1 moderate positive sample to evaluate five kits repeatability and imprecision, and the CV value was less than 5%. LoD is the lowest analyte concentration. Daan kit has the lowest LoD than other four kits. We evaluated the LoD of these four kits SARS-CoV-2 is a beta coronavirus belonging to the family of Coronaviruses with at least 70% similarity in genetic sequence to SARS-CoV-1. 6,14 ORF1/a gene is unique to SARS-CoV-2, whereas E gene was pan-Sarbecovirus detection and the expected cross-reactivity with the SARS-CoV-1 E gene. 15 There were no cross-reactions by the five kits for ORF1ab and N gene with six human coronaviruses and four respiratory pathogens. Considering a variety of clinical scenarios, multiplex PCR-based assay that can simultaneously detect many pathogens including SARS-CoV-2 may become a regular detection method. 16 The study has some limitations. First, the 20 clinical specimens from SARS-CoV-2 confirmed patients were coming from same region. Geographical underrepresentation may influence the results. Second, we used the same virus RNA extraction kit for five kits. It might affect some kits. Third, the 20 positive clinical specimens were all nasopharyngeal swabs specimens. While a nasopharyngeal swab is the preferred testing method, further investigation to evaluate RT-PCR kit performance using other specimen types is needed. 8 In conclusion, we believe that five RT-PCR kits included in this study have good diagnostic performance and are useful tools for the routine diagnosis. We advise that the high-risk populations, such as people who are close contact with COVID-19 patients, patients during later stages of the infection, people who have fever but have not been detected and caregivers, should be detected by low LoD kits. Return of the Coronavirus: 2019-nCoV A Novel Coronavirus Emerging in China -Key Questions for Impact Assessment Overcoming the bottleneck to widespread testing: a rapid review of nucleic acid testing approaches for COVID-19 detection Clinical characteristics of 140 patients infected with SARS-CoV-2 in Wuhan The implications of preliminary screening and diagnosis: Clinical characteristics of 33 mild patients with SARS-CoV-2 infection in Hunan Comparison of seven commercial RT-PCR diagnostic kits for COVID-19 Limits of Detection of Six Approved RT-PCR Kits for the Novel SARS-coronavirus-2 (SARS-CoV-2). Clinical chemistry, hvaa099. Advance online publication Comparison of nasopharyngeal and oropharyngeal swabs for SARS-CoV-2 detection in 353 patients received tests with both specimens simultaneously Potential False-Negative Nucleic Acid Testing Results for Severe Acute Respiratory Syndrome Coronavirus 2 from Thermal Inactivation of Samples with Low Viral Loads Laboratory verification of an RT-PCR assay for SARS-CoV-2 Comparative performance of four nucleic acid amplification tests for SARS-CoV-2 virus SARS-CoV-2 viral load in sputum correlates with risk of COVID-19 progression Quantitative Detection and Viral Load Analysis of SARS-CoV-2 in Infected Patients Properties of Coronavirus and SARS-CoV-2 Direct in Vivo RNAi Screen Unveils Myosin IIa as a Tumor Suppressor of Squamous Cell Carcinomas Evaluation of a multiplex PCR assay for detection of respiratory viruses and Mycoplasma pneumoniae in oropharyngeal swab samples from outpatients