key: cord-0749084-vzf3j29m
authors: Owen, Sophie I.; Williams, Christopher T.; Garrod, Gala; Fraser, Alice J.; Baldwin, Lisa; Brown, Lottie; Byrne, Rachel L.; Collins, Andrea M.; Cubas-Atienzar, Ana I.; de Vos, Margaretha; Edwards, Thomas; Escadafal, Camille; Ferreira, Daniela M.; Fletcher, Tom; Hyder-Wright, Angela; Kay, Grant A.; Kontogianni, Konstantina; Mason, Jenifer; Menzies, Stefanie; Mitsi, Elena; Planche, Tim; Sacks, Jilian A.; Taylor, Joseph; Todd, Stacy; Tully, Caroline; Cuevas, Luis E.; Adams, Emily R.
title: Twelve lateral flow immunoassays (LFAs) to detect SARS-CoV-2 antibodies
date: 2021-12-11
journal: J Infect
DOI: 10.1016/j.jinf.2021.12.007
sha: 53dfc144e673c42c3e0face7226b3156c398ae3a
doc_id: 749084
cord_uid: vzf3j29m

BACKGROUND: There are an abundance of commercially available lateral flow assays (LFAs) that detect antibodies to SARS-CoV-2. Whilst these are usually evaluated by the manufacturer, externally performed diagnostic accuracy studies to assess performance are essential. Herein we present an evaluation of 12 LFAs. METHODS: Sera from 100 SARS-CoV-2 reverse-transcriptase polymerase chain reaction (RT-PCR) positive participants were recruited through the FASTER study. A total of 105 pre-pandemic sera from participants with other infections were included as negative samples. RESULTS: At presentation sensitivity against RT-PCR ranged from 37.4-79% for IgM/IgG, 30.3-74% for IgG, and 21.2-67% for IgM. Sensitivity for IgM/IgG improved ≥21 days post symptom onset for 10/12 tests. Specificity ranged from 74.3-99.1% for IgM/IgG, 82.9-100% for IgG, and 75.2-98% for IgM. Compared to the EuroImmun IgG enzyme-linked immunosorbent assay (ELISA), sensitivity and specificity ranged from 44.6-95.4% and 85.4-100%, respectively. CONCLUSION: There are many LFAs available with varied sensitivity and specificity. Understanding the diagnostic accuracy of these tests will be vital as we come to rely more on the antibody status of a person moving forward, and as such manufacturer-independent evaluations are crucial.

As of May 2021, there have been over 100,000,000 confirmed cases of COVID-19 worldwide, however the total number of cases is much higher (1). This is due to, amongst other reasons, the lack of diagnostic testing worldwide in the first wave of the pandemic, the continued difficulties in testing in some lowermiddle income countries and the number of asymptomatic infections that continue to go undetected (2, 3) .

Detecting antibody responses to SARS-CoV-2 therefore could prove vital, both for understanding previous exposure on an individual level, but also at community and regional levels.

During SARS-CoV-2 infection, IgM and IgG titres begin to increase from around 10 days post-symptom onset with IgM titres generally declining earlier than IgG (4, 5) . The reference standard for detecting an antibody response, either IgM, IgG, or both, to SARS-CoV-2 is the immunoassay, usually either an enzyme-linked immunosorbent assay (ELISA) or chemiluminescence assay. Many immunoassays have been developed and commercialised for SARS-CoV-2 which are highly sensitive and specific (6) . The process however is time-consuming, expensive, requires specialist laboratory equipment and trained professionals to perform.

To this end, large numbers of lateral-flow immunoassays (LFAs) have been developed that detect IgM and IgG responses. These are simple tests which require only a drop of blood and the addition of buffer to give a result in 10-15 minutes. Results are easy to interpret; if a test line appears the participant is considered positive, if no test line appears the participant is considered negative. LFAs are easy to massproduce and are affordable making them ideal for mass-testing of populations, rapid identification of antibody response in travel situations, or for home-testing. Those that detect both IgM and IgG are also able to give an indication of when that person was likely infected, due to the dynamics of the antibody response.

Many LFAs are commercially available, and externally performed diagnostic accuracy studies, independent of the manufacturer, are required to generate robust performance data. Here, we present the evaluation of twelve LFAs and describe their diagnostic accuracy in a cohort of 100 confirmed SARS-CoV-2 positive participants with varying disease severity and 105 samples from participants confirmed as negative or collected pre-pandemic. This study forms part of a larger initiative to generate and share independent performance data on COVID-19 tests coordinated by FIND, the global diagnostics alliance.

Research samples were provided with informed written consent. Participants were recruited through the 

A total of 142 serum/plasma samples from 125 SARS-CoV-2 RT-qPCR -positive participants were used in this study to assess the sensitivity of the LFAs. One hundred serum/plasma samples from RT-qPCRpositive participants were used for each LFA due to the limited quantity of some serum samples. Full details are given in Table S1 . Briefly, 24 patients presenting at the Liverpool University Hospitals NHS A total of 105 SARS-CoV-2 negative serum samples were used to assess specificity of the LFAs. See Table S2 for full details. These consisted of 84 pre-pandemic sera collected from individuals diagnosed with influenza A (n=20), tuberculosis (TB) (n=10), human immunodeficiency virus (HIV) (n=10), TB/HIV (n=10), dengue virus (n=10), parasitic diseases (n=12), human coronavirus 229E (n=10) and human coronavirus OC43 (n=2). A panel of pre-pandemic plasma from participants with non-COVID-19related fever (n=20) were provided by FIND and an additional quality assurance sample (n=1).

The Anti-SARS-CoV-2 ELISA (IgG) kit (EI 2606-9601 G) (EuroImmun, Germany) was used to screen all serum samples for the presence of anti-SARS-CoV-2 IgG, as per the manufacturer's instructions.

Samples with an OD value greater than the calibrator were considered positive, samples with OD value lower than the calibrator were considered negative.

Twelve LFAs (Table 1) were evaluated according to manufacturer's instructions. Briefly, 10-20 µl serum was required depending on the test, followed by 2-3 drops of buffer. Results were read independently by two people; if there was any disagreement a third person acted as a tiebreaker. Full details are given in Table 1 . Of the 12 tests, 11 detected IgM and IgG separately, with only Beijing Wantai giving a 'total antibody' result. All tests were CE-IVD marked.

Sensitivity was calculated against RT-qPCR confirmed SARS-CoV-2 infections including sensitivity when stratified by days post-symptom onset. Specificity was calculated against RT-qPCR confirmed SARS-CoV-2 negative samples or samples collected pre-pandemic. Sensitivity was then calculated against RT-qPCR confirmed SARS-CoV-2 infections also positive by IgG ELISA. Specificity was calculated against IgG ELISA negative samples. Percentage agreement and Cohen's Kappa statistic against IgG ELISA were calculated (8) . Data analyses were carried out in MedCalc for Windows, version 19.8 (MedCalc Software, Ostend, Belgium).

Sensitivity of the LFAs against RT-qPCR ranged from 37. 4 Figure 1 ). The sensitivity for an IgM/IgG response increased in 10 out of 12 tests at >21 days post-symptom onset, with a mean increase of 15.0% (Table 2, Figure 1 ).

The sensitivity for IgG increased >21 days post-symptom onset, with a mean increase of 16.5% (Table 2, Figure 1 ). For IgM, sensitivity was higher ≤21 days post-symptom onset in six of the LFAs and higher at >21 days post-symptom onset for five tests ( Figure 1 ). For all tests, except for Beijing Wantai which is a combined IgM/IgG only, and Shenzhen Bioeasy where specificity for IgM/IgG and IgG were identical, IgG alone had a higher specificity than IgM/IgG (Table 2, Figure 1 ). Four tests reported 100% specificity for IgG (Table 2) . Nine

LFAs had a higher IgG specificity than IgM (Table 2, Figure 1 ).

Of the 142 participants positive by RT-qPCR, 90 (63.4%) were positive by IgG ELISA. Of the 105 prepandemic samples, 85 were tested by IgG ELISA due to sample availability, of which 82 (96.5%) were negative by IgG ELISA. Compared to IgG ELISA, LFA sensitivity was found to range between 44.6% and 95.4% (Table 3) . Shanghai Kehua had the highest sensitivity at 95.4% (Table 3 ). Compared to IgG ELISA, LFA specificity was found to range between 85.4% and 100% ( Table 3 ). The highest agreement between IgG measured by LFA and IgG measured by ELISA was seen with Shanghai Kehua (Table 4 ).

There is a plethora of LFAs available on the market today, all purporting to offer high sensitivity and specificity, but often without rigorous, manufacturer-independent evaluations. In this study, we evaluated In this study, no test met the clinical sensitivity requirements of >98% (95% CI: 96-100%) on samples collected ≥20 days post-symptom onset laid out in the target product profile (TPP) published by the UK government (10). However, our data are calculated on fewer than 200 confirmed positive cases as specified by the TPP (10). As expected with an earlier decline in IgM titres, fewer LFAs had improved sensitivity for IgM >21 days post-symptom onset. It is important to note that the samples used in this study were collected before the roll out of any COVID-19 vaccine.

The large variation in performance in LFAs reported here is in accordance with other evaluations (9) . The variations in diagnostic accuracy may in part be due to the antigen used to detect SARS-CoV-2 antibodies. The two main immunogenic antigens of SARS-CoV-2 are the nucleocapsid and the surface spike protein, split into domains S1 and S2, with the receptor-binding domain in S1. S1 is thought to be the most specific, with low-level cross-reactivity demonstrated for S2 and nucleocapsid (11) . It is one or a combination of these antigens that are used for serological testing.

information is key to testing during vaccine-rollout, with two vaccines, Pfizer-BioNTech and Moderna, containing mRNA encoding spike proteins to elicit an anti-spike immune response. LFAs that detect a response to spike antigens should prove useful for detecting both prior exposure to SARS-CoV-2, but also There is still no established 'gold-standard' serological test for SARS-CoV-2. The EuroImmun IgG ELISA was chosen at the time as it was one of the only CE-marked ELISA assays. We therefore use RT-qPCR as the main reference test in this study. Previous studies have reported false positives with the EuroImmun IgG ELISA, and we report here 3/105; one from a HIV patient in Nigeria in 2018, one from a patient with dengue virus in Brazil in 2015 and one with an individual with human coronavirus 229E in the UK in 2019. These were excluded from the ELISA/LFA specificity analysis. Cross-reactivity was seen in an evaluation of the EuroImmun Anti-SARS-CoV-2 NCP ELISA carried out in Nigeria in a negative control panel in which 50.2% of participants had the P. falciparum HRP2 antigen (12) . The specificity of the EuroImmun ELISA in this study was 96.5%.

LFAs can be helpful in measuring exposure of a community to SARS-CoV-2, particularly in areas where testing of symptomatic individuals was not and is not readily available. Manufacturer-independent evaluations provide helpful data as to the accuracy of LFAs, as there is a large variation in the performance characteristics of these assays . Further evaluations are needed following the commencement of vaccination campaigns to evaluate the use of LFAs in vaccinated individuals.

The authors have no conflicts of interest to declare. Table 2 . Sensitivity and specificity of the 12 LFAs. Sensitivity was calculated using SARS-CoV-2 RT-qPCR positive sera/plasma and specificity determined on pre-pandemic sera/plasma. Table 3 . Sensitivity and specificity of LFAs against IgG ELISA. 

Charting the challenges behind the testing of COVID-19 in developing countries: Nepal as a case study

A systematic review of asymptomatic infections with COVID-19

Temporal profiles of viral load in posterior oropharyngeal saliva samples and serum antibody responses during infection by SARS-CoV-2: an observational cohort study

Serum IgA, IgM, and IgG responses in COVID-19

Comparative assessment of multiple COVID-19 serological technologies supports continued evaluation of point-of-care lateral flow assays in hospital and community healthcare settings

Self-sampling of capillary blood for serological testing of SARS-CoV-2 by COVID-19 IgG ELISA. medRxiv

The Measurement of Observer Agreement for Categorical Data

Target Product Profile: antibody tests to help determine if people have recent infection to SARS-CoV-2: Version 2 -GOV.UK [Internet

Immunologic Testing for SARS-CoV-2 Infection from the Antigen Perspective

Cross-reactivity of two SARS-CoV-2 serological assays in a malaria-endemic setting

We would like to thank all those who participated in the FASTER study for their involvement in this research. We would also like to thank the teams from Liverpool University Hospitals National Health