key: cord-0735620-yin6njxd
authors: Dema, Emily; Copas, Andrew J; Clifton, Soazig; Conolly, Anne; Blake, Margaret; Riddell, Julie; Boso Perez, Raquel; Tanton, Clare; Bonell, Chris; Sonnenberg, Pam; Mercer, Catherine H; Mitchell, Kirstin R; Field, Nigel
title: Methodology of Natsal-COVID Wave 1: a large, quasi-representative survey with qualitative follow-up measuring the impact of COVID-19 on sexual and reproductive health in Britain
date: 2022-03-28
journal: Wellcome Open Res
DOI: 10.12688/wellcomeopenres.16963.2
sha: 630422a9f2a52503241876945e9619111b04dbc7
doc_id: 735620
cord_uid: yin6njxd

Background: Britain’s National Surveys of Sexual Attitudes and Lifestyles (Natsal) have been undertaken decennially since 1990 and provide a key data source underpinning sexual and reproductive health (SRH) policy. The COVID-19 pandemic disrupted many aspects of sexual lifestyles, triggering an urgent need for population-level data on sexual behaviour, relationships, and service use at a time when gold-standard in-person, household-based surveys with probability sampling were not feasible. We designed the Natsal-COVID study to understand the impact of COVID-19 on the nation’s SRH and assessed the sample representativeness. Methods: Natsal-COVID Wave 1 data collection was conducted four months (29/7-10/8/2020) after the announcement of Britain’s first national lockdown (23/03/2020). This was an online web-panel survey administered by survey research company, Ipsos MORI. Eligible participants were resident in Britain, aged 18-59 years, and the sample included a boost of those aged 18-29. Questions covered participants’ sexual behaviour, relationships, and SRH service use. Quotas and weighting were used to achieve a quasi-representative sample of the British general population. Participants meeting criteria of interest and agreeing to recontact were selected for qualitative follow-up interviews. Comparisons were made with contemporaneous national probability surveys and Natsal-3 (2010-12) to understand bias. Results: 6,654 participants completed the survey and 45 completed follow-up interviews. The weighted Natsal-COVID sample was similar to the general population in terms of gender, age, ethnicity, rurality, and, among sexually-active participants, numbers of sexual partners in the past year. However, the sample was more educated, contained more sexually-inexperienced people, and included more people in poorer health. Conclusions: Natsal-COVID Wave 1 rapidly collected quasi-representative population data to enable evaluation of the early population-level impact of COVID-19 and lockdown measures on SRH in Britain. Although sampling was less representative than the decennial Natsals, Natsal-COVID will complement national surveillance data and Natsal-4 (planned for 2022).

• Sexual and reproductive health (SRH) remains important during the COVID-19 pandemic.

• However, a lack of attention to this aspect of health and well-being, as well as the challenges of addressing this sensitive topic have meant that the impacts of the pandemic for sexual behaviour and SRH have largely been ignored.

• Non-pharmaceutical interventions to reduce SARS-CoV-2 transmission meant that in-person, household-based probability methods were not feasible during this time.

• Incorporating learning from the decennial Natsal survey, Natsal-COVID demonstrates the feasibility of obtaining valuable public health data from a web-panel using quota-sampling and weighting to provide a quasirepresentative national sample.

• Natsal-COVID is unique in the pragmatic collection of population-level SRH data to inform policy and practice responses to the pandemic in a timely manner and supplement data from surveillance systems, service users, and the decennial Natsal study

Sexual and reproductive health (SRH) is integral to wider health and well-being 1 . Measuring and monitoring SRH is crucial in normal times, and remains so during the COVID-19 pandemic 2 . However, none of the large national surveys undertaken to assess the impact of COVID-19 and associated restrictions included questions about sexual behaviour or SRH. This partly reflects the challenges of asking about sensitive and sometimes stigmatising behaviours, and also longstanding failure to prioritise this aspect of individual and public health 3 . Furthermore, existing cohort studies in Britain do not focus on SRH, so it was not possible to use these as was done for some other areas of health (e.g., UK Household Longitudinal Study) 4 .

The largest and most comprehensive population-based studies of SRH in Britain are the decennial National Surveys of Sexual Attitudes and Lifestyles (Natsal). These have developed rigorous methods to obtain high-quality data, including optimising sampling, data collection methods, and question wording. Consequently, findings from Natsal have informed SRH policy and practice in Britain and internationally since 1990 5 . However, due to the risks of COVID-19, lockdown restrictions, and the need for timely data, Natsal's methods (i.e., householdbased interviewing and probability sampling using the Postal Address Files (PAF)) were not feasible at this time. The pilot for the fourth decennial Natsal survey was paused until 2021 due to these restrictions, allowing the team to conduct the Natsal-COVID study. The Natsal-COVID study sought to capitalise on the experience of Natsal to understand the impact of COVID-19 on SRH in Britain and took a pragmatic approach to achieve the best quality possible under the circumstances. We prioritised a large-scale national quota sample with online data collection that could be achieved rapidly and at relatively low cost, which facilitates multiple waves of data collection to monitor changes during and after the pandemic. We also undertook qualitative interviews to gain greater contextual understanding and to provide insights into the thoughts, feelings, and behaviours of participants reporting three types of experience in the survey, chosen for their public health importance. The reported experiences were: sex with someone outside the household during lockdown; difficulty accessing SRH services; and relationship difficulties A second wave conducted in March-April 2021 captures behaviour and outcomes one-year after the first national lockdown in Britain (due to report in July 2021).

In response to increasing COVID-19 infections, the UK government announced the first national lockdown on 23 March 2020, which meant individuals were asked to stay at home except for essential shopping, medical care, and exercise 6,7 . From mid-May, restrictions were gradually eased. However, some form of restrictions and physical distancing requirements remained throughout the summer ( Figure 1 ). Wave 1 of Natsal-COVID aimed to understand early changes in SRH service use and need, sexual behaviours, and relationships during this time. It was designed to capture experiences during the four months following the beginning of the first national lockdown in the UK, including a period of subsequent partial easing of restrictions ( Figure 1 ). This paper describes the methods used in the Wave 1 of Natsal-COVID and assesses the representativeness of the data.

The target sample size was primarily based on the detection of change in key behavioural and other outcomes over time between the first (Wave 1) and follow-up (Wave 2) samples with comparisons within-person based on those completing both Waves. We can also consider 'cross-sectional' analyses within each Wave in selecting the sample size. We set an initial sample size target of 6,000 participants aged 18-59 years old for Wave 1 and assumed that ≥2,000 of each gender (4,000 in total) would complete follow-up (those from Wave 1 who do not complete Wave 2 are 'replaced' by new participants for Wave 2). We assumed a design effect of <1.33 giving an effective sample size of ≥1,500 of each gender (3,000 in total) completing follow-up. To detect a 5% significance level (two-sided testing) of change over time in a less common behaviour for each gender (such as partner change), we anticipated over 80% power to detect change as small as 2 vs. 3% (RR 1.5), provided the correlation over time within individuals was moderate as expected (>0.25). For less common behaviours reported primarily by one gender (e.g., emergency contraception use), power was anticipated to be >80% for change as small as 2 vs. 3.5% (RR 1.75).

The new version of this paper includes minor edits throughout the paper in response to reviewer comments. We have also updated Figure 1 to clarify the study timeframe. Since data from wave 1 of Natsal-COVID is now available via the UK Data Service, we also provide details for accessing the dataset.

For a common behaviour, such as sex in the past four weeks by gender, power was anticipated as >80% even for small changes such as 50 vs. 55% (RR 1.1), provided the correlation over time was >0.1. This sample size of 6,000 participants per Wave, assuming a design effect of <1.33, provides an effective sample size >2,200 of each gender at each Wave. For a 'crosssectional' analysis of change between Waves within each gender, including all participants from both Waves and conservatively ignoring the correlation between responses from the same individuals, the design provides 90% power to detect differences in outcomes such as 2.0% vs. 3.6% and 50% vs 55%. The core sample therefore included 6,000 people aged 18-59 years with an additional 'boost sample' of 500 people aged 18-29 years to ensure a sample of 2,000 people aged 18-29 was achieved across whole sample. The boost was added because the burden of SRH needs falls predominantly on young people.

Quotas were set for the core and boost samples based on gender, age, region, and social grade to achieve a quasi-representative sample of the general population aged 18-59 years. The quotas for gender, age, and region used ONS mid-year estimates for 2019 8 . The quotas for social grade used census data from 2011 (as mid-year or more recent estimates were not available for this measure) 9 , which used data for those aged 16-59 years rather than 18-59 years due to data availability.

We obtained ethics approval from University of Glasgow MVLS College Ethics Committee (reference 20019174) and

London School of Hygiene and Tropical Medicine Research Ethics committee (reference 22565). Participants provided consent to participate via an online consent form prior to the start of the survey.

The Natsal-COVID questionnaire was adapted from the Natsal-3 questionnaire 10,11 . and informed by development work for Natsal-4, which was paused until mid-2021 due to the pandemic. Questions specific to the COVID-19 pandemic were included, many of which were drawn from other major COVID-19 studies 12-14 . Some question wording was adapted for an online mode of delivery. For example, timeframes were changed, pop-up boxes were used to show definitions of key terms (e.g., oral, anal, and vaginal sex), and the survey length and complexity were reduced to improve completion without an interviewer being present. No formal validation testing was conducted for the whole questionnaire, but many questions were based on previously validated wording (e.g. Natsal-3), which was updated to reflect the time frames needed in the context of the pandemic. Some measures which were included in the Natsal-COVID questionnaire, such as the generalised anxiety disorder two item (GAD-2) and patient health questionnaire two item (PHQ-2) scales, have been described and validated elsewhere 15, 16 .

Natsal-COVID included questions on sexual activity and relationships over different timeframes, including since the start of the first national lockdown, intimate contact with people outside of their household since lockdown, as well as relationship quality and sexual function, and SRH service use and unmet need (Box 1). The full questionnaire is available at the study website and will be made available as Extended data. As in the decennial Natsal, the Natsal-COVID survey utilised routing to minimise participants being asked questions irrelevant to their own situation and experiences. In line with other COVID-19 surveys and due to the lack of a baseline immediately prior to the pandemic, the questionnaire included questions on perceived changes compared to the three months prior to lockdown. For further information about the questionnaire, interested researchers may contact the Natsal team (natsal@ucl.ac.uk). 

Survey data were collected from 29 July 2020 to 10 August 2020. The online panels are run with stringent recruitment and quality-control processes to ensure individuals can only join once, are not excessively sampled for surveys, and so remain engaged. 164,074 panellists were contacted via email to participate in Natsal-COVID ( Figure 2 About half of participants completed the survey on a laptop or desktop (49%), and the rest via smartphone (45%) or tablet (6%). Median survey length was 10 minutes; the interquartile range was 7 minutes to 14 minutes.

Panellists receive small incentives to participate in Ipsos MORI surveys in the form of points, which can be redeemed for modest rewards and entry into sweepstake draws. Panellists who do not qualify for a survey (i.e., do not progress beyond the screening questions) also receive a small number of points for their willingness to participate.

Natsal-COVID was inclusive in its approach to gender, including in response options and questionnaire routing. Although panel quotas were based on the proportions of participants identifying as men and women (i.e., not including 'in another way'), the survey asked participants to self-report gender identity, with the options being 'man', 'woman', or 'in another way' (e.g., non-binary), and sex assigned at birth (options being 'male', 'female', or 'prefer not to say'). Sixty-one participants were classified as 'trans' (a derived variable) where their reported sex at birth was different to their reported gender identity, including 24 trans men, 14 trans women, and 24 who identified in another way ( Figure 3 ), giving an overall percentage of 0.8% in the weighted sample (Table 1) . Where data are presented for men and women, these estimates include trans men and trans women, respectively. The participants identifying 'in another way' were included in analyses where the denominator is everyone but were not included in denominators for men or women.

Towards the end of fieldwork, some quotas were relaxed to ensure enough people from harder-to-recruit groups were included. Initially, regional quotas were relaxed to increase the number of young men, and over this period, the numbers in the lower social grades were also increased. Subsequently, all other quotas were relaxed to increase the number of participants in Scotland and Wales. Overall, this meant that target quotas (age, gender, region, and social grade) exceeded 90% in the final sample, with the exception of regional quotas for Wales (82%) and Greater London (87%).

Weighting was used to achieve a quasi-representative sample of the population of Britain by gender, age, region, social grade, ethnicity, and sexual identity. Weighting targets were based on ONS 2019 mid-year census estimates 8 for age, gender, and region CI=confidence intervals.

*Data are presented for individuals aged 18-59 from England, Scotland, and Wales. Sexual identity comparisons ("don't know" are excluded from the table but included in denominator) come from a report, which reports sexual identity for individuals aged 16+ in the entire UK.

and 2011 census figures 9 for social grade and ethnicity. The initial weights (including gender, age, region, social grade, and ethnicity) did not include sexual identity but we observed over-representation of non-heterosexual individuals (unweighted; Table 1 ), which we would expect to introduce bias within a study on sexual health. A final weight was created to include sexual identity, based on the Annual Population Survey (APS) 2018, which was a probability sample survey of those aged 16 years and older in the UK 17 .

Ipsos MORI calculated weights using the random iterative method based on regression analysis (see Ipsos MORI Technical Report). The weight calculations were repeated until the weights were sufficiently close to the target for all factors; this convergence occurred on the fourth iteration. The weighting efficiency for the weights was 87.0%.

The Natsal-COVID sample was compared with the following probability sample surveys to assess its representativeness ( Table 1 and As expected, due to quota sampling and weighting, the Natsal-COVID sample was similar to the external datasets for gender, age, region, ethnicity, and sexual identity (Table 1) . Data on social grade in the external datasets (APS and HSE) were not available. Non-heterosexual-identifying participants were over-represented in the Natsal-COVID unweighted sample (men, 13.3%; women, 10.7%) but the weighted proportions of non-heterosexual participants were broadly comparable between Natsal-COVID (men, 3.9%; women, 3.7%) and APS (men, 3.0%; women, 2.6%) (APS data includes participants aged 16+, with no upper age limit, so differences in non-heterosexual identities between unweighted samples may be partially explained by the upper age limit of 59 years in the Natsal-COVID sample).

The weighted Natsal-COVID sample was similar to APS/HSE for other demographic variables, including urban rural classification ( Table 2 ). The Natsal-COVID sample under-represented participants reporting their legal marital status as being 'married' (40.5%) compared to 2019 APS (47.5%) and overrepresented those in poorer health compared to the 2018 HSE sample (fewer Natsal-COVID participants reported 'very good' health, and more reported 'fair' health). However, there were no differences in the proportion reporting a limiting long-term illness or disability.

The Natsal-COVID sample also had a higher education level than the HSE sample (Table 2 ). Only 4.3% of Natsal-COVID participants reported no qualification, compared to 11.3% of HSE participants, though this should be interpreted with caution given differences in response options between the two surveys.

The Natsal-COVID sample included a higher proportion reporting no previous sexual experience (not necessarily involving genital contact) (9.5%) compared with Natsal-3 (1.3%) ( Table 2) . Natsal-COVID also had a much smaller proportion reporting sex in the past year (i.e., sexually-active) (69.2%) compared to Natsal-3 (91.5%). However, when restricted to sexually-experienced participants (at least one partner, lifetime), the distribution of partner numbers (of any gender) over their lifetime was similar between Natsal-3 and Natsal-COVID. Men in Natsal-COVID (6.3%) were equally likely to report any previous same-sex experience as men in Natsal-3 (5.7%). Women in Natsal-COVID (5.4%) were slightly less likely to report any previous same-sex experience compared to those in Natsal-3 (7.3%). The proportion of sexually-active men reporting at least one new partner in the past year was higher among the Natsal-COVID sample (30.0%) compared to Natsal-3 (25.8%). This proportion was similar among women in Natsal-COVID (18.6%) and Natsal-3 (19.7%).

Semi-structured qualitative follow-up interviews were conducted with 45 selected Natsal-COVID participants to explore three types of experience reported in the quantitative survey, chosen for their public health importance: (group 1) sexual contact with someone living outside their household, (group 2) needing, but being unable to access SRH services, and (group 3) increased arguments and reduced support from their partner since lockdown. Interviewees were selected from the 771 survey participants who had agreed to follow up, provided contact details, and met the criteria of reporting one (or more) of these experiences. We used qualitative purposive sampling to: ensure variation by age, gender, ethnicity, and region; select for attributes of interest (e.g., oversampling of women in group two to reflect higher use of SRH services compared to men); and maximise information (e.g., selecting individuals who met more than one criterion).

The research team contacted, by phone or email, 143 individuals who had expressed willingness to take part in further research with information on the qualitative study and to confirm their eligibility. Of these, 67 individuals were uncontactable (either due to no response after three attempts or an incorrect phone number), 20 declined to take part, six were not eligible, two were interested but were not able to make an interview time, and three expressed interest, but the quota had already been filled. Interested participants were emailed the study documentation and given time to decide if they would like to take part. Interviews were conducted between October and November 2020. Participants were offered interview by phone or video, with all but two participants opting for phone interviews. Participants completed and returned a consent form via email, which was followed up with an audio recording prior to the start of the interview where participants verbally confirmed their consent to take part.

Interviews lasting between 36 to 92 minutes (mean duration 65 minutes) were conducted by three trained qualitative CI=confidence intervals.

*Data are presented for individuals aged 18-59 from England, Scotland, and Wales. Sexual identity comparisons ("don't know" are excluded from the table but included in denominator) come from a report, which reports sexual identity for individuals aged 16+ in the entire UK in 2018.

interviewers (DR, KJM, RBP). Fieldnotes reflecting on interview content were documented after each interview. Participants were offered a £30 e-voucher token of appreciation. Interview guides (to be made available as Extended data) covered socio-demographic characteristics, lockdown household composition, views on COVID-19 and social restriction measures, and impact of COVID-19 on personal life, in addition to questions exploring the three topics outlined above (Box 2). Audio recordings were transcribed intelligent verbatim (transcriber discretion to omit utterances that don't add any meaning) by transcription agencies under contract. Transcripts were reviewed by the study team to check for accuracy as well as to develop familiarity with participant accounts. Transcripts were anonymised and entered into Nvivo 12 20 . Data were analysed thematically using a framework approach 21 . An individual researcher led the analysis for each of the three groups but the coding frameworks, coding decisions, and emerging themes were discussed between analysts during regular analysis meetings. The analytical approach will be described in each paper reporting the results. 

Natsal-COVID is a large, national study that was rapidly undertaken in Britain at a time when data were urgently required to understand the impact of the pandemic for SRH clinical and public health policy and decision-making. Due to restrictions on movement and meeting indoors, this was also at a time when in-person household-based probability sample surveys were not feasible nor sufficiently rapid. Natsal-COVID fills an important evidence gap because other COVID-19 studies have not addressed the population-level impact of COVID-19 on sexual behaviour and SRH, or conducted qualitative interviews to more thoroughly understand SRH challenges in the general population. The Natsal-COVID study also benefits from methodological elements developed by the Natsal team in consultation with stakeholders 22 over several decades to inform the most rigorous and ethical approaches when asking about highly sensitive behaviours and experiences. We have demonstrated that it is possible to achieve a large-scale quasi-representative sample within 12 days of fieldwork during a period of intense social disruption.

Natsal-COVID is not a probability sample and is therefore not truly representative of the general population 23 . Instead, we relied upon quota-based sampling and weighting to achieve a webpanel sample that is quasi-representative of the general population in Britain in terms of age, gender, region, social grade, and sexual identity. There are well described sources of bias in web-panel surveys 24, 25 , which might affect the results and interpretation. Previous studies have demonstrated that nonprobability web-panel surveys, such as Natsal-COVID, are less representative of the general population than probability surveys, such as the decennial Natsal survey 24 . One important source of bias in web-panels is that they include only those with access to the internet. According to an Ofcom report, although 87% of the UK population older than 16 years reported using the internet in 2019 26 , those in lower social grades and older adults were less likely to do so. Therefore, among our sample age range of 18-59 years, we anticipate a higher proportion of individuals with internet access.

Although quota sampling is likely to be more representative of the population than self-selecting or convenience samples 27 , which have been the primary methods used to study sexual health in the COVID-19 pandemic, there are also limitations in the use of quota sampling for the Natsal-COVID study. Quota samples and web-panel samples are particularly susceptible to non-response and residual bias due to self-selection 27 . Comparisons with external probability surveys and with Natsal-3 show the Natsal-COVID sample to be generally similar for key sociodemographic characteristics and sexual behaviours. However, we demonstrated appreciable sampling bias for several important characteristics that remained after applying weighting. Natsal-COVID over-represented individuals with higher education levels, which is consistent with previous findings for web samples [23] [24] [25] . Although Natsal-COVID had a higher proportion of participants reporting 'fair health' compared to HSE, differences here might be due to the impact of the pandemic on perceived health status, and it was noteworthy that there were not major differences in the proportions reporting limiting long-term illness or disability. Natsal-COVID also had fewer participants reporting sex in their lifetime than we would expect from Natsal-3 (albeit that Natsal-3 was undertaken ten years ago). This could be due to differences in the question wording and participants' understanding, a mode effect (i.e., online versus inperson), or sampling bias, and Natsal-COVID may therefore include more people at lower risk of adverse SRH outcomes (e.g., people with no previous sexual experience). Prior to weighting, Natsal-COVID had more men identifying as non-heterosexual men compared to 2019 APS. The finding of over-representation of non-heterosexual men in Natsal-COVID (prior to weighting) is consistent with previous web-panel surveys 24 . However, weighted percentages of non-heterosexual men were comparable between Natsal-COVID and 2019 APS 28 . So, while the findings of Natsal-COVID are likely to be broadly generalisable, its prevalence estimates should be treated with particular caution.

The inclusion of semi-structured interviews facilitated understanding of phenomena of interest from the perspective of participants. Understanding the meaning of experiences and context in which they take place, can facilitate interpretation of associations identified in quantitative data, as well as surfacing key issues not asked about in the survey 29 . Recruitment of participants from the survey sample has several benefits including being able to identify individuals with very specific experiences of interest. The large sample frame also enabled sufficient variation on key characteristics such age, gender, and region 30 . We were unable to triangulate across survey and interview responses due to regulations stipulating sharing of only de-identified data by Ipsos MORI. The fact that interviews were subsequent to the survey allowed participants time for reflection, and this may facilitate more candid reporting 31 . Building rapport is also important to enabling detailed and candid accounts. It was a drawback that we were unable to conduct in-person interviews due to pandemic restrictions. Although we anticipated that most participants would opt for video interviews (in order to see the interviewer), most actually preferred to use the telephone. We did not ask participants directly, but it seems likely that this was due to privacy concerns.

In conclusion, the Natsal-COVID Wave 1 study can be considered quasi-representative of the British population, and its data can be used to quantify and better understand the initial impacts of the COVID-19 pandemic on sexual behaviour and SRH in Britain following the start of the first national lockdown in the UK in March 2020. A second Wave of data collection will enable us to capture impacts throughout the year following the first national lockdown in the UK. Although not as representative as the decennial Natsal study, the data from the Natsal-COVID study are already informing SRH policy and service delivery during and after the COVID-19 pandemic.

Datasets 

This study provides a technical report for the design, methodology and implementation of an internet-based survey conducted as part of Natsal (National Survey of Sexual Atittudes and Lifestyles). Because of COVID-19, Natasl could not conduct fieldwork as usual but did a highquality pivot to online data collection. Unlike in the United States where fielding of the parallel National Survey of Family Growth did not occur during COVID, this new NATSAL data offers the potential for gaining important insights into the impact of the pandemic on sexual and reproductive health and well-being.

Data are not presently public available at the time of this publication; if they are now, this could be updated.

While I think this is an excellent manuscript that will be valuable to many readers, I have a few suggestions for improvements. These include the following: Box 1-is it correct that no information about pregnancy desires, experiences, or outcomes was collected? If so, why not?

1.

Explain why those under age 18 were not included.

Please provide citations for this assertion "Web-panel methodology precludes calculation of any response rates because panelists are invited in waves and selected based on quotas." AAPOR provides guidance for this so I'm not sure it's a correct statement or needs further clarification.

The difference between this sample and other national studies in the proportion reporting no previous sexual experience is striking and I feel needs more discussion/investigation. Is this universal or an issue for specific subgroups? How did your question wording compare? Are there reasons to think this could represent some real social change over time?

Please say more about why so many respondents in the qualitative study were uncontactable.

This new paper is making waves and may be worth reflecting on. Bradley, V.C., Kuriwaki, S., Isakov, M., Sejdinovic, D., Meng, X.L. and Flaxman, S., 2021. Unrepresentative big surveys significantly overestimated US vaccine uptake. Nature, 600(7890), pp.695-700. 1 6. I don't understand why you couldn't connect people's qual and survey interviews? Why could Ipsos MORI connect them and then not share the identifying information? This seems like a missed opportunity.

Could point to other studies that also shifted to video or phone interviews during COVID. This is a new and exciting methodology that Natsal is one example of.

In the conclusion, please add more detailed information on how you expect the data to be used--what questions do you hope this data will help to answer? Also, the first sentence of the final paragraph states as fact what I think is hope --that his study WILL 'enable us to quantify and better understand...": 9. This study provides a technical report for the design, methodology and implementation of an internet-based survey conducted as part of Natsal (National Survey of Sexual Atittudes and Lifestyles). Because of COVID-19, Natasl could not conduct fieldwork as usual but did a high-quality pivot to online data collection. Unlike in the United States where fielding of the parallel National Survey of Family Growth did not occur during COVID, this new NATSAL data offers the potential for gaining important insights into the impact of the pandemic on sexual and reproductive health and well-being.

RESPONSE: Thank you for your helpful comments, which we have addressed below. We feel these have really helped to improve the paper.

were uncontactable, half were not reached within three attempts and for the other half, the phone number was incorrect. We have updated the text to include this information.

Yes. The authors have provided a clear explanation of the methods they have used to develop their study, achieving an optimal balance of detail and brevity where appropriate, including referring the reader to existing documentation already available to enhance their description. There are some areas where a replicator may need additional details from the authors (e.g. the developmental history of specific survey questions) but these are best obtained from directly contacting the authors, rather than including all possible permutations in the paper.

If any results are presented, are all the source data underlying the results available to ensure full reproducibility? ○ N/A. Study results are not presented other than the number of types of participants, by relevant characteristics, required to generate weighted data and check the representativeness of their samples against external datasets.

Are the conclusions about the method and its performance adequately supported by the findings presented in the article? ○ Yes, all conclusions are highly appropriate, except for the final conclusion, which I have commented on below. In addition, I have made a few extra comments that the authors might find useful in enhancing their paper for clarity.

It is unclear why qual participants were included so it would be helpful to briefly indicate the reason for this.No further comments by my side ○ 'and inform policy' -this paper, which focuses on reporting study methodology, does not provide any evidence about this issue so perhaps it should be deleted from this abstract.

These are clear and appropriate.

A clear, balanced rationale has been provided for the importance of the Natsal-COVID study.

○ It would be helpful to specify the dates of wave 2 and the 'types of experience' used to sample qual participants. It would be useful to update on the report which was due in July 2021, if possible.

○ Figure 1 is helpful, but the 'Natsal' row needs greater explanation. Specifically, it is not immediately unclear what 'to 1 yr prior to interview' and 'period of reporting' mean.

It is important to clarify early on that Natsal 4 was planned for 2020 but paused due to the pandemic -perhaps in paragraph 2 of the background -which is why Natsal-COVID was conducted (if this is indeed the case).

A clear description of the target sample size, using design effect, power, change detected and risk ratio is provided for the overall sample size and the 18-29 boost sample. Although the process described is logical, I always think it is useful to refer readers to relevant texts/papers to confirm the literature underpinning these analyses (although I note that it is usual to miss this out). 'Quota group' -it might be helpful to signal that this issue is discussed in later in the paper.

○ 'Median interview length' -it would be helpful to replace the phrase 'interview' here with 'survey' (e.g. survey completion time') as here I assume the authors are referring to the quant survey data completion and not the later qualitative interviews? Referring to both as interviews is likely to confuse these two different parts of the data collection.

Gender has been dealt with in an appropriate way herein, and clearly explained which is commendable. In addition, a reasonable proportion of trans folk have been included in the study which is a good achievement for this kind of general population survey.

It is important to have a couple of sentences to clarify what quotas were originally set up for the study as this is not clearly mentioned earlier. It might be useful to reference the technical reports mentioned later to help with this. Relaxation of quota sampling is clearly explained as is weighting.

This issue is clearly explained, and appropriately and comprehensively conducted with reference to appropriate external datasets. Over-representation of LGBTQI+ folk is to be expected given the focus on sexual health and sampling frame, though it might be worth clarifying why this is in the paper. The over-representation of folk reporting 'fair' as opposed to 'very good' health is to be expected during a global pandemic as are lower proportions reporting 'sex in the past year' after 4 months of lockdown and prohibited extra-domestic liaisons. Overall, a clear case is made for the representativeness of the sample given the potential for real behavioural changes engendered by the pandemic itself.

It might be useful to clearly locate this quota sampling as qualitative purposive quota sampling in the text.

○ 'women in group 2' -I do not understand what being in 'Group 2' means? This appears in Box 2 but at this stage the reader has yet to be introduced to this information. Perhaps the first sentence of this section needs to clarify this. ○ Qualitative analysis: I am unclear why the method underpinning the qualitative analysis is presented in this paper but not the quantitative analysis. I can understand that each quantitative driven paper will likely have its own analytical plan, meaning this cannot be useful detailed herein. However, I would have thought this was also possible for the qual data, so to provide details for one but not the other in this paper feels rather unbalanced to me and I'd opt to provide neither herein.

'for web' -perhaps add in 'samples' before the references? ○ It would be helpful to explain why not conducting f2f interviews was a drawback, since this is not necessarily the case for telephone interviews, which themselves have many advantages (e.g. better perceived anonymity, greater participant and researcher convenience etc.), which may also explain why video interviews are not preferred.

○ 'In conclusion, the Natsal-COVID Wave 1 study has enabled us to quantify and better understand the initial impacts of the COVID-19 pandemic on sexual behaviour and SRH in Britain following the start of the first national lockdown in the UK in March 2020.' Though I have no doubt that these data will allow this, you cannot conclude this from the information presented in this paper. Accordingly, I would recommend you focus your conclusion on the analysis presented herein; that it is possible to collect data using online panel sampling methods, which are convincingly comparable to representative samples conducted using offline methods (etc.).

Are sufficient details provided to allow replication of the method development and its use by others? Yes ' We also undertook qualitative interviews to gain greater contextual understanding and to provide insights into the thoughts, feelings, and behaviours of participants reporting three types of experience in the survey, chosen for their public health importance. ' We feel this is complete and don't propose to alter the manuscript.

'and inform policy' -this paper, which focuses on reporting study methodology, does not provide any evidence about this issue so perhaps it should be deleted from this abstract.

○ RESPONSE: We will delete "and inform policy" from the abstract.

Key messages: These are clear and appropriate.

Background: A clear, balanced rationale has been provided for the importance of the Natsal-COVID study.

○ It would be helpful to specify the dates of wave 2 and the 'types of experience' used to sample qual participants. It would be useful to update on the report which was due in July 2021, if possible.

○ RESPONSE: After the sentence "We also undertook qualitative interviews to gain greater contextual understanding and to provide insights into the thoughts, feelings, and behaviours of participants reporting three types of experience in the survey, chosen for their public health importance", we will add "The reported experiences were: sex with someone outside the household during lockdown; difficulty accessing SRH services; and relationship difficulties." We have also specified the months for Wave 2 data collection (March-April 2021). Figure 1 is helpful, but the 'Natsal' row needs greater explanation. Specifically, it is not immediately unclear what 'to 1 yr prior to interview' and 'period of reporting' mean.

○

Participants were asked about behaviours and outcomes during the period from the start of lockdown (23 March 2020) . We refer to this 4-month period as 'period of reporting'. Some outcomes for the 'past year' were also asked. We have updated Figure 1 so that this is clearer.

It is important to clarify early on that Natsal 4 was planned for 2020 but paused due to the pandemic -perhaps in paragraph 2 of the background -which is why Natsal-COVID was conducted (if this is indeed the case).

brevity and depth has been achieved here and the actual survey is available online. It is always desirable for specific details of all survey questions' development/origins to be available (and herein the C19 questions in particular). However, it is not feasible to provide such details for every item for such a large, detailed questionnaire. Therefore I agree with the level of detail provided herein and that further details should be restricted to future publications, where necessary. Perhaps the authors could suggest that further details about the origins/development of specific items could be provided upon request? RESPONSE: The following sentence will be added. "For further information about the questionnaire, interested researchers may contact the Natsal team (natsal@ucl.ac.uk)" 'In conclusion, the Natsal-COVID Wave 1 study has enabled us to quantify and better understand the initial impacts of the COVID-19 pandemic on sexual behaviour and SRH in Britain following the start of the first national lockdown in the UK in March 2020.' Though I have no doubt that these data will allow this, you cannot conclude this from the information presented in this paper. Accordingly, I would recommend you focus your conclusion on the analysis presented herein; that it is possible to collect data using online panel sampling methods, which are convincingly comparable to representative samples conducted using offline methods (etc.).

○ RESPONSE: We have updated the conclusion paragraph to focus on the analysis and quasirepresentative nature of the sample, which can be analysed for broader SRH impact.

No competing interests to declare.

Comparison data from Annual Population Survey

White includes all those who identify as White English, Welsh, Scottish, Northern Irish, British, Irish, Gypsy or Irish Traveller, or from any other White background 4. Mixed ethnicity includes those who identify as White and Black African, White and Black Caribbean, White and Asian, or any other mixed or multiple ethnic background 5. Asian includes those who identify as Indian, Pakistani, Bangladeshi, Chinese or from any other Asian background 6. Black includes those who identify as African, Caribbean, or from any other Black background 1. Comparison data from Annual Population Survey

) NVQ4 / HNC / HND / Bachelor's degree or similar, and (7) NVQ5 or post-graduate diploma. A 3-category education variable, based on a variable used by 2018 Health Survey for England (HSE), including "no qualification

Comparison data from Natsal-3 5. All respondents 6. All respondents reporting at least one sexual partner in their lifetime (vaginal, anal, oral sex, or other genital contact)

All respondents reporting at least one sexual partner in the past year (vaginal, anal, oral sex, or other genital contact)

Reference Source 2. World Health Organization: Maintaining essential health services: operational guidance for the COVID-19 context

Changes in sexual attitudes and lifestyles in Britain through the life course and over time: findings from the National Surveys of Sexual Attitudes and Lifestyles (Natsal)

PubMed Abstract | Publisher Full Text | Free Full Text 6. Institute for Government: Timeline of UK coronavirus lockdowns

UCL COVID-19 Social Study: Understanding the psychological and social impact of the pandemic

The Patient Health Questionnaire-2: validity of a two-item depression screener

Screening for anxiety disorders with the GAD-7 and GAD-2: a systematic review and diagnostic metaanalysis

QSR International Pty Ltd: NVivo (Version 12)

Qualitative data analysis: the framework approach

The fourth National Survey of Sexual Attitudes and Lifestyles (Natsal-4): Responses to the Consultation on Questionnaire Content

Design of a survey of sexual behaviour and attitudes in the British population: Scoping review

Nonprobability Web Surveys to Measure Sexual Behaviors and Attitudes in the General Population: A Comparison With a Probability Sample Interview Survey

Ofcom: Online Nation 2020 Summary report

GSR quota sampling guidance

Using the Web to Collect Data on Sensitive Behaviours: A Study Looking at Mode Effects on the British National Survey of Sexual Attitudes and Lifestyles

Qualitative Methods for Health Research

Unrepresentative big surveys significantly overestimated US vaccine uptake

How can we facilitate reliable reporting in surveys of sexual behaviour? Evidence from qualitative research. Cult Health Sex

SN: 8649. 2020

London School of Hygiene and Tropical Medicine. Centre for Sexual and Reproductive Health Research

We want to thank the study participants and Reuben Balfour (Ipsos MORI). We thank Dr Caisey Pulford, Senior Surveillance and Prevention Scientist at Public Health England, who designed a Figure that we used to create Figure 1 . We also thank David Reid (UCL) and Dr Karen J Maxwell (University of Glasgow) for their work as qualitative interviewers on this study. ○ RESPONSE: We will add the following sentence in paragraph 2. ""The pilot for the fourth decennial Natsal survey was paused until 2021 due to these restrictions, allowing the team to conduct the Natsal-COVID study."

Data are not presently public available at the time of this publication; if they are now, this could be updated.RESPONSE: Data are now available, and the Data availability statement has been updated accordingly.While I think this is an excellent manuscript that will be valuable to many readers, I have a few suggestions for improvements. These include the following: Box 1-is it correct that no information about pregnancy desires, experiences, or outcomes was collected? If so, why not?1.

In this first wave of the Natsal-COVID study, data on pregnancy desires, experiences, or outcomes were not collected due to the short reporting timeframe (4 months from the start of the first British lockdown). The second wave of Natsal-COVID (methods of which will be reported separately) does include questions about pregnancy outcomes and pregnancy planning.Explain why those under age 18 were not included. 1.

Please provide citations for this assertion "Web-panel methodology precludes calculation of any response rates because panelists are invited in waves and selected based on quotas." AAPOR provides guidance for this so I'm not sure it's a correct statement or needs further clarification.

RESPONSE: AAPOR provide guidance for reporting precision in non-probability samples, and has guidance for reporting response rates in probability web surveys, but not response rates for nonprobability surveys. Elsewhere, some have suggested 'metrics' for non-probability web surveys (https://doi.org/10.1093/poq/nfn065). Taken together, we prefer to retain the sentence as written.The difference between this sample and other national studies in the proportion reporting no previous sexual experience is striking and I feel needs more discussion/investigation. Is this universal or an issue for specific subgroups? How did your question wording compare? Are there reasons to think this could represent some real social change over time?

RESPONSE: Sexual experience was determined based on reported numbers of partners in participants' lifetime. Question wording was similar between Natsal-3 and Natsal-COVID. We also note that lifetime partner numbers were similar between Natsal-COVID and Natsal-3 when restricted to sexually-experienced participants. However, the studies are separated in time by over 10 years, and we are unable to account for secular changes in sexual behaviours that may have occurred in this time. It is therefore unclear whether this reflects a true population-level change, including due to the pandemic, is due to differences in the methodologies, or a combination of these factors. Natsal-4, starting fieldwork in late 2022, is well-placed to given further insight into these questions. We have not made further alterations to the paper.Please say more about why so many respondents in the qualitative study were uncontactable.

This new paper is making waves and may be worth reflecting on. Bradley, V.C., Kuriwaki, S., Isakov, M., Sejdinovic, D., Meng, X.L. and Flaxman, S., 2021. Unrepresentative big surveys significantly overestimated US vaccine uptake. Nature, 600(7890), pp.695-700.1

RESPONSE: Thank you for flagging this paper to us. We were really interested to read it. Although this provides insight into the quality of different online survey methodologies, we note that it only applies to one outcome (vaccine uptake). We take from it that the Ipsos MORI approach is better than convenience sampling. However, the complex and sometimes sensitive information collected in Natsal-COVID limits how much we can infer for our study. Nevertheless, we've cited this paper in our discussion.I don't understand why you couldn't connect people's qual and survey interviews? Why could Ipsos MORI connect them and then not share the identifying information? This seems like a missed opportunity.

RESPONSE: Apologies that this was not clear. In fact, we were able to connect people's qual and quant survey data, and we have modified the paper to emphasize that this was possible.Could point to other studies that also shifted to video or phone interviews during COVID. This is a new and exciting methodology that Natsal is one example of.1.

interviews. However, the Natsal-COVID quantitative survey was delivered solely online. We will report on the Natsal-4 experiences in other methodological papers in due course.In the conclusion, please add more detailed information on how you expect the data to be used--what questions do you hope this data will help to answer? Also, the first sentence of the final paragraph states as fact what I think is hope --that his study WILL 'enable us to quantify and better understand...":

RESPONSE: Thanks for this comment. We have modified the discussion to express this as an aspiration. The many and varied uses of the Natsal-COVID data make it difficult to list every way in which the data might be used. Is the rationale for developing the new method (or application) clearly explained? Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

This study reports the methodology, quota sampling and weighting for an internet panel-based iteration of the decennial national probability Natsal (National Survey of Sexual Attitudes and Lifestyles) study, itself the world-leading study of sexual behaviours for a national population. C19 restrictions prevented the normal sampling methodology of Natsal during 2020; this paper critically discusses the new methodology developed to address this.

Is the rationale for developing the new method (or application) clearly explained? ○ Yes. This study reports the methodology, quota sampling and weighting for an internet panelbased iteration of the national probability Natsal study, itself the world-leading study of sexual behaviours for a national population. C19 restrictions prevented the normal sampling methodology of Natsal during 2020, so this paper is required to explain the new methodology developed. Moreover, this methodology represents the state of the art in terms of developing potentially representative national samples during C19, and so the rationale for developing this method is clear. It is also directly relevant to other situations where researchers need to assess the generalisability of their internet-based samples to the wider population. Is the description of the method technically sound? Approval Status:The methods used are clearly and appropriately described within the paper and Wellcome Open Research is an appropriate forum for this paper. Accordingly, I approve this paper subject to very minor revisions, which I have detailed below.RESPONSE: Thank you for these comments, which we have responded to below. We feel these have really helped to improve the paper.Article Summary: This study reports the methodology, quota sampling and weighting for an internet panelbased iteration of the decennial national probability Natsal (National Survey of Sexual Attitudes and Lifestyles) study, itself the world-leading study of sexual behaviours for a national population. C19 restrictions prevented the normal sampling methodology of Natsal during 2020; this paper critically discusses the new methodology developed to address this.Specific questions for Article type: Is the rationale for developing the new method (or application) clearly explained? ○ Yes. This study reports the methodology, quota sampling and weighting for an internet panel-based iteration of the national probability Natsal study, itself the world-leading study of sexual behaviours for a national population. C19 restrictions prevented the normal sampling methodology of Natsal during 2020, so this paper is required to explain the new methodology developed. Moreover, this methodology represents the state of the art in terms of developing potentially representative national samples during C19, and so the rationale for developing this method is clear. It is also directly relevant to other situations Sample Design: A clear description of the target sample size, using design effect, power, change detected and risk ratio is provided for the overall sample size and the 18-29 boost sample. Although the process described is logical, I always think it is useful to refer readers to relevant texts/papers to confirm the literature underpinning these analyses (although I note that it is usual to miss this out).○ RESPONSE: Thanks for the comment. Such papers will be referred to in subsequent work that arises from the Natsal-COVID study.It might be helpful to clarify that the 2011 census is the most recently available in the UK at this time.○ RESPONSE: We will update the sentence to the following: "The quotas for social grade used census data from 2011 (as mid-year or more recent estimates were not available for this measure)."Is there a typo in the first sentence 'and McNemar's test' ○ RESPONSE: This typo will be deleted.Ethics: These issues are clearly explained. Ethics obtained from U of G and LSHTM and therefore appropriate. 'Quota group' -it might be helpful to signal that this issue is discussed in later in the paper.○ RESPONSE: The following will be added after "quota group": (described below)'Median interview length' -it would be helpful to replace the phrase 'interview' here with 'survey' (e.g. survey completion time') as here I assume the authors are referring to the quant survey data completion and not the later qualitative interviews? Referring to both as interviews is likely to confuse these two different parts of the data collection. ○ RESPONSE: We will change "interview" to "survey".Gender: Gender has been dealt with in an appropriate way herein, and clearly explained which is commendable. In addition, a reasonable proportion of trans folk have been included in the study which is a good achievement for this kind of general population survey. with reference to appropriate external datasets. Over-representation of LGBTQI+ folk is to be expected given the focus on sexual health and sampling frame, though it might be worth clarifying why this is in the paper. The over-representation of folk reporting 'fair' as opposed to 'very good' health is to be expected during a global pandemic as are lower proportions reporting 'sex in the past year' after 4 months of lockdown and prohibited extra-domestic liaisons. Overall, a clear case is made for the representativeness of the sample given the potential for real behavioural changes engendered by the pandemic itself.Qualitative sampling: It might be useful to clearly locate this quota sampling as qualitative purposive quota sampling in the text. Qualitative analysis: I am unclear why the method underpinning the qualitative analysis is presented in this paper but not the quantitative analysis. I can understand that each quantitative driven paper will likely have its own analytical plan, meaning this cannot be useful detailed herein. However, I would have thought this was also possible for the qual data, so to provide details for one but not the other in this paper feels rather unbalanced to me and I'd opt to provide neither herein.

We only provide very basic information on qualitative analysis that is common across all qualitative outputs. We believe this will be of interest to qualitative researchers. Given that the paper as a whole is unbalanced towards the quantitative survey, we would prefer to keep this in.Discussion: 'for web' -perhaps add in 'samples' before the references? ○ RESPONSE: The text has been updated.It would be helpful to explain why not conducting f2f interviews was a drawback, since this is not necessarily the case for telephone interviews, which themselves have many advantages (e.g. better perceived anonymity, greater participant and researcher convenience etc.), which may also explain why video interviews are not preferred.○ RESPONSE: In sexual health research, there are many advantages to f2f surveys. Response rates are much higher, we are able to randomly select one individual per address, interviewer rapport encourages participation and facilitates privacy, participant safety is maintained, questionnaire