key: cord-325455-e464idc0
authors: Atchison, Christina; Pristerà, Philippa; Cooper, Emily; Papageorgiou, Vasiliki; Redd, Rozlyn; Piggin, Maria; Flower, Barnaby; Fontana, Gianluca; Satkunarajah, Sutha; Ashrafian, Hutan; Lawrence-Jones, Anna; Naar, Lenny; Chigwende, Jennifer; Gibbard, Steve; Riley, Steven; Darzi, Ara; Elliott, Paul; Ashby, Deborah; Barclay, Wendy; Cooke, Graham S; Ward, Helen
title: Usability and acceptability of home-based self-testing for SARS-CoV-2 antibodies for population surveillance
date: 2020-08-12
journal: Clin Infect Dis
DOI: 10.1093/cid/ciaa1178
sha: 
doc_id: 325455
cord_uid: e464idc0

BACKGROUND: This study assesses acceptability and usability of home-based self-testing for SARS-CoV-2 antibodies using lateral flow immunoassays (LFIA). METHODS: We carried out public involvement and pilot testing in 315 volunteers to improve usability. Feedback was obtained through online discussions, questionnaires, observations and interviews of people who tried the test at home. This informed the design of a nationally representative survey of adults in England using two LFIAs (LFIA1 and LFIA2) which were sent to 10,600 and 3,800 participants, respectively, who provided further feedback. RESULTS: Public involvement and pilot testing showed high levels of acceptability, but limitations with the usability of kits. Most people reported completing the test; however, they identified difficulties with practical aspects of the kit, particularly the lancet and pipette, a need for clearer instructions and more guidance on interpretation of results. In the national study, 99.3% (8,693/8,754) of LFIA1 and 98.4% (2,911/2,957) of LFIA2 respondents attempted the test and 97.5% and 97.8% of respondents completed it, respectively. Most found the instructions easy to understand, but some reported difficulties using the pipette (LFIA1: 17.7%) and applying the blood drop to the cassette (LFIA2: 31.3%). Most respondents obtained a valid result (LFIA1: 91.5%; LFIA2: 94.4%). Overall there was substantial concordance between participant and clinician interpreted results (kappa: LFIA1 0.72; LFIA2 0.89). CONCLUSION: Impactful public involvement is feasible in a rapid response setting. Home self-testing with LFIAs can be used with a high degree of acceptability and usability by adults, making them a good option for use in seroprevalence surveys.

Lateral flow immunoassays (LFIA) offer a rapid point-of-care (POC) approach to novel coronavirus (SARS-CoV-2) antibody testing. While LFIAs may not currently be accurate enough for individual-level clinical decisions (1, 2) , they are valuable as a public health tool. On a population level, by conducting seroprevalence surveys through widespread random sampling of the general public, and by adjusting for the sensitivity and specificity characteristics of the LFIA used, it is possible to estimate the levels of past infection with SARS-CoV-2 in the community (3) .

However, testing hundreds of thousands of people would be impractical if it required a blood sample to be drawn followed by processing in a laboratory. One solution is to use self-sampling and selftesting in the home with participants reporting results to the researchers. However, there is limited understanding of public acceptability and usability of these LFIAs in the home setting, as most are currently designed as POC tests performed by healthcare professionals.

Self-sampling and self-testing are widely used in healthcare for monitoring, for example in diabetes management (4) , and for diagnostics, for example for HIV (5, 6) . There are many advantages in terms of uptake, cost, patient activation and scale (4, 6) , but also potential disadvantages in relation to validity, usability and practicality which should be explored (6, 7) . Usability research on HIV selftesting has generally found good acceptability, the devices easy to use and high validity in interpretation of self-reported test results (7) (8) (9) . However, these HIV test kits were designed for selfsampling and self-testing and went through several iterations before designs were appropriate for home use and therefore the same levels of acceptability and usability for home-based self-testing for SARS-CoV-2 antibody using LFIAs cannot be assumed. As part of the REal-time Assessment of Community Transmission (REACT) programme (10), we evaluated the acceptability and usability of LFIAs for use in large seroprevalence surveys of SARS-CoV-2 antibody in the community.

A c c e p t e d M a n u s c r i p t 5

We evaluated two LFIAs with different usability characteristics from five LFIAs being validated in parallel in our laboratory-based study (11). Both LFIAs required a blood sample from a finger-prick and produced a self-read test result after 10 or 15 minutes. LFIA1 (Guangzhou Wondfo Biotech Co Ltd) was a cassette-based system containing a "control" indicator line and a "test" indicator line (for detection of combined IgM and IgG antibodies). LFIA2

(Fortress Orient Gene Biotech Co Ltd) was a cassette-based system containing a "control" indicator line and separate indicator lines for IgM and IgG ( Figure 1 ).

In early May 2020 we carried out rapid, iterative public involvement and a pilot usability study including an online forum with four discussion groups (n=37), a study of LFIA1 test use with volunteers (n=44) and a broader public sample (n=234), and a nested observation and interview study (n=25). Further details on the methods, including how we recruited participants from our existing involvement networks, are available online (Supplement, S1).

The test kits dispatched in the pilot study included one test cassette, one button-activated 28G lancet and a 2ml plastic pipette, alongside an instruction booklet also containing a weblink to an instructional video. Based on findings from the pilot study, for the larger population-based usability study, the lancet and pipette were replaced with two pressure-activated 23G (larger) lancets and a smaller 1ml plastic pipette, respectively. The design and language in the instructional booklet and video were changed and an alcohol wipe was also included in the kit.

A c c e p t e d M a n u s c r i p t 6 In late May 2020 we carried out a larger population-based usability study of a representative sample of the adult population (aged 18 years and over) in England. We used addresses from the Postal Address File to draw a random sample of 30,000 households in England to which study invitation letters were sent. We allowed up to four adults aged 18 and over in the household to register for the study. Self-testing LFIA kits were then posted to each registered individual. On completion of the test, participants recorded their interpretation of the result as part of an online survey, with the option of uploading a photograph of the test result. Reminder letters were sent to participants who had not completed the online survey or uploaded a photograph within 10 days of test kits being dispatched.

Metrics to evaluate usability and acceptability were based on the HIV self-testing literature (5, 6, 8) and were measured as the percentage of participants responding to specific closed questions in the online survey. The questionnaire used is available as an online supplement (Supplement, S2). The main outcome was usability of the LFIA kits. This was defined as a participant's ability to complete the antibody test, and how easy or difficult it was to understand the instructions and complete each step in the process. Acceptability was measured in terms of people consenting to and using the provided self-test, and the proportion who reported they would be willing to repeat a self-administered finger-prick antibody test in the future. 

Analyses were conducted in Stata (version 15.0, StataCorp, Texas, USA).

Data obtained from the questionnaires on acceptability and usability were summarised by counts and descriptive statistics, and comparisons were made between LFIA1 and LFIA2 using Pearson's chisquared test.

Multivariate regression was used to identify sociodemographic factors independently associated with the proportion of participants conducting the test that achieved a valid result. Variables that appeared to be associated (p<0.05) in the unadjusted analyses were considered in the adjusted analyses. Adjusted odds ratios (aOR) and 95% confidence intervals (CI) were estimated. Associations with a p-value <0.05 in the adjusted analyses were considered statistically significant.

Agreement between participant-interpreted and clinician-interpreted result for test outcomes (negative, IgG positive, invalid, unable to read) was assessed using the Fleiss Kappa statistic. A c c e p t e d M a n u s c r i p t 8 (11). Therefore, we used the same operational definition of positivity for LFIA2. Participants were informed in the instructions to consider IgM results as negative.

The study obtained research ethics approval from the South Central-Berkshire B Research Ethics Committee (IRAS ID: 283787).

Overall, 315 members of the public contributed feedback during the involvement and pilot. This led to changes in the design and language used in the instructional video and booklet, the type and number of lancets and the size of pipette in the LFIA kits (further details available online, Supplement S1).

For the national study, 25,000 household invitation letters were sent, and 17,411 individuals registered for the study from 8,508 households. Due to the maximum number of kits we had available for the study, 14,400 participants were selected at random from those who registered.

Thus, 10,600 LFIA1 kits were distributed, with 8,754 individual user surveys completed (82.6% response rate), and 3,800 LFIA2 kits were distributed, with 2,957 individual user surveys completed (77.8% response rate). Most commonly, two adults participated per household (Table 1 ). Baseline characteristics of study participants are shown in Table 1 . The median age of participants across LFIA1 and LFIA2 was 51.0 years (range 18 to 95). There were some differences between LFIA1 and LFIA2 participants by ethnicity, region and household size.

Acceptability of self-testing was high ( A c c e p t e d M a n u s c r i p t 9 As in the pilot, most respondents were willing to perform another finger-prick antibody test in the future (LFIA1: 98.3%; LFIA2: 97.0%). Only a minority preferred to do the antibody test in a clinical care or community setting than at home. As with the pilot study, respondents with children showed a high willingness to perform the antibody test on them, and this proportion increased with the age of the children (Table 2) .

In the pilot study most people (86.5%, 225/260) who attempted the test managed to complete it.

However, significant usability issues were identified, including challenges with the lancet to obtain a blood drop and the pipette to transfer the blood to the sample well. The problems with the lancet led to some participants using alternative objects to draw blood, including pins and sewing needles, while others opened the lancet casing to access the blade. Some people reported minor problems putting buffer into the buffer well. This led to the inclusion of two lancets and changes to the instructions for the national study.

In the national study, almost all participants who attempted the antibody test reported completing it (97.5% for LFIA1 and 97.8% for LFIA2) ( Table 2) . Reasons for not completing the test are shown in the table. Of LFIA1 participants who reported damaging the test, the majority reported either accidentally removing the entire lid off the buffer bottle and spilling the solution all over the test cassette or putting the blood and buffer in the wrong well. For LFIA2, few participants damaged the test and they all reported putting the blood and buffer in the wrong well.

About one in four participants asked someone to help them to administer the test. Most found the instructions easy to understand (Figures 3), but as in the pilot, participants reported some difficulties in performing the test. For LFIA1, difficulties with using the pipette were reported by 17.7%

(1,512/8,521) of participants. In addition, 10.6% (908/8,556) had difficulties applying the blood to A c c e p t e d M a n u s c r i p t 10 the sample well ( Figure 3 ). Therefore, for LFIA2 the instructions were changed to omit the use of the pipette and instead directly transfer blood from the finger-prick site to the sample well. However, participants still found creating a blood drop from the finger-prick site (23.2%; 664/2,862) and then applying the blood to the well (31.3%; 894/2,858) difficult. LFIA2 was deployed after LFIA1 because there was a delay in arrival of LFIA2 from the supplier. This differential timing in dispatch of the kits had the unexpected benefit of allowing us to make iterative changes to the instructions. Overall, 7.4% of LFIA1 and 4.8% of LFIA2 participants reported an invalid result (Table 3 ). There was some variation in the proportion of participant-reported invalid results using LFIA1 by age and gender. The higher the number of participants registered for the study in the same household, the lower the odds of the participant reporting an invalid result. No sociodemographic factors were associated with a participant-reported invalid result using LFIA2 (Table 4 ).

After adjusting for sociodemographic differences between LFIA1 and LFIA2 participants, there was no difference between LFIA1 and LFIA2 in terms of being able to read the result (1.1% vs. 0.81%; aOR 0.76 (95% CI: 0.48-1.2); p 0.24). But a lower percentage of invalid test results were reported by LFIA2 participants (7.9% vs. 4.8%; aOR 0.64 (95% CI: 0.53-0.77); p <0.001). Table 5 shows concordance between participant and clinician interpreted results in the national study. For LFIA1, there was substantial agreement overall (kappa 0.72 (95% CI: 0.71-0.73); p<0.001), however there were important differences: while there was 100.0% agreement for results reported as negative, and 98.5% agreement for invalid results, a clinician confirmed only 62.8% of participantreported positives. Visible reasons (from the photograph) were insufficient blood volume to cover the bottom of the sample well, or insufficient movement of the blood and buffer solution across the result window. In addition, the clinician was able to interpret the results of all but four out of 66 A c c e p t e d M a n u s c r i p t 11 (6.1%) tests from the photographs of results participants reported as "unable to read". The four results unreadable by the clinician were because blood had leaked out of the sample well and obscured the result window. Of participant-reported unable to read results, 78.8% were clinicianinterpreted as invalid (for all these tests, the control and "test" lines were both absent). Table 5 ). The clinician could not interpret the results from 10 photographs reported as readable results by participants, reasons included blurred photographs and shadowing obscuring the indicator lines.

Overall, we found that self-testing with the two LFIA kit designs used in this study was highly acceptable among adults living in England. High acceptability of in home self-testing is in keeping with self-sampling and self-testing studies in diabetes management (4), and HIV diagnostics (5, 6) .

The majority of participants who attempted the test successfully completed it despite some continued difficulties with using the pipette (LFIA1), creating a drop of blood from the finger-prick site (LFIA2) and applying the blood to the sample well (LFIA1 and LFIA2). Based on these findings, we proportion of invalid tests for LFIA2 could reflect the better designed buffer bottle (which was a significant issue for LFIA1), better performance characteristics of LFIA2 over LFIA1 (e.g. easier movement of the blood and buffer solution across the result window), or improved ability to get sufficient blood in the sample well using the direct blood transfer technique over using a pipette.

Participants' ability to obtain a valid test result using LFIA1 varied marginally by age and gender and increased with the number of participants registered for the study in the same household. The latter observation is not surprising as observing another household member performing the test is likely to improve the performance of others in the same household. No sociodemographic differences in obtaining a valid result were found for LFIA2.

Of note, about one in four participants reported that they had help administering the test, irrespective of LFIA used. This could put individuals living alone at a disadvantage in terms of usability. However, comparing those that had help to those that did not, we found no difference in ability to complete the test (97.7% vs. 98.2%; p 0.08) or reported invalid results (6.1% vs. 7.0%; p 0.06).

Overall, there was good agreement between self-reported results and those reported by a clinician.

Therefore, our findings broadly support self-reporting of home-based test results using LFIAs. But the public and individual health impact of misinterpreting a test result that is negative but read as positive is a concern as an individual could falsely conclude that they have antibodies for SARS-CoV-2 and may change their behaviour as a result. To mitigate against this, and given the scientific Our study is original because focusing on the acceptability and usability of LFIAs for self-testing for SARS-CoV-2 antibody in a home-based setting has not been done at such scale in the general population. It provides an attractive solution for conducting large seroprevalence surveys. The study has, however, some limitations. Study participants may not be representative of the general adult population of England. However, we had data on the England population profile (2011 census data (13)), as well as the study registration profile and survey completion profile of the study participants which gave us an indication of response bias. Our sample was broadly similar to the England population profile. In addition, the usability study was conducted in parallel with our laboratorybased study of performance characteristics of LFIAs. As such, we did not know the accuracy of the LFIAs chosen for the usability study at the time, or whether either would perform well enough in the laboratory to be considered for the large national seroprevalence study planned as part of the REACT programme (10). However, given that the majority of commercially available LFIAs have a similar cassette-based design and test result read out to LFIA1 or LFIA2 we were confident that our results would be generalisable and applicable to whichever LFIA was finally selected. Findings from our laboratory-based study, including the performance characteristics of LFIA1 and LFIA2 are forthcoming (11).

Overall, our study has demonstrated that home-based self-testing LFIAs for use in large communitybased seroprevalence surveys of SARS-CoV-2 antibody are both acceptable and feasible. Although this study identified a few usability issues, these have now been addressed. LFIA2, Fortress Orient Gene, has been selected for a large national seroprevalence study as part of the REACT programme.

This decision was based on criteria including the usability and acceptability determined in this study, A c c e p t e d M a n u s c r i p t 21 years where asked whether they would carry out the test on children of that age living in their households. Denominator is th e number of participants who reported having children of that age living in their household. M a n u s c r i p t 23 

Antibody testing for COVID-19: A report from the National COVID Scientific Advisory Panel

Evaluation of nine commercial SARS-CoV-2 immunoassays

Prevalence of SARS-CoV-2 in Spain (ENE-COVID): a nationwide, population-based seroepidemiological study

Self-monitoring of blood glucose in type 2 diabetes: recent studies

Acceptability, feasibility, and individual preferences of blood-based HIV self-testing in a population-based sample of adolescents in Kisangani, Democratic Republic of the Congo

Reliability of HIV rapid diagnostic tests for self-testing compared with testing by health-care workers: a systematic review and meta-analysis

Interferences and Limitations in Blood Glucose Self-Testing: An Overview of the Current Knowledge

Usability assessment of seven HIV self-test devices conducted with lay-users in

We thank key collaborators on this work --Ipsos MORI: Stephen Finlay, John Kennedy, Duncan