key: cord-280491-tarb3mu7
authors: Wu, Yin; Kwakkenbos, Linda; Henry, Richard S.; Tao, Lydia; Harb, Sami; Bourgeault, Angelica; Carrier, Marie-Eve; Levis, Brooke; Sun, Ying; Bhandari, Parash Mani; Carboni-Jiménez, Andrea; Gagarine, Maria; He, Chen; Krishnan, Ankur; Negeri, Zelalem F.; Neupane, Dipika; Mouthon, Luc; Bartlett, Susan J.; Benedetti, Andrea; Thombs, Brett D.; Culos-Reed, Nicole; El-Baalbaki, Ghassan; Hebblethwaite, Shannon; Patten, Scott; Varga, John; Bustamante, Laura; Duchek, Delaney; Ellis, Kelsey; Rice, Danielle; Dyas, Laura; Fortuné, Catherine; Gietzen, Amy; Guillot, Geneviève; Lewis, Nancy; Nielsen, Karen; Richard, Michelle; Sauvé, Maureen; Welling, Joep
title: Validation of the COVID-19 fears questionnaires for chronic medical conditions: A scleroderma patient-centered intervention network COVID-19 cohort study
date: 2020-10-08
journal: J Psychosom Res
DOI: 10.1016/j.jpsychores.2020.110271
sha: 
doc_id: 280491
cord_uid: tarb3mu7

Objective Fear associated with medical vulnerability should be considered when assessing mental health among individuals with chronic medical conditions during the COVID-19 pandemic. The objective was to develop and validate the COVID-19 Fears Questionnaire for Chronic Medical Conditions. Methods Fifteen initial items were generated based on suggestions from 121 people with the chronic autoimmune disease systemic sclerosis (SSc; scleroderma). Patients in a COVID-19 SSc cohort completed items between April 9 and 27, 2020. Exploratory factor analysis (EFA) and item analysis were used to select items for inclusion. Cronbach's alpha and Pearson correlations were used to evaluate internal consistency reliability and convergent validity. Factor structure was confirmed with confirmatory factor analysis (CFA) in follow-up data collection two weeks later. Results 787 participants completed baseline measures; 563 of them completed the follow-up assessment. Ten of 15 initial items were included in the final questionnaire. EFA suggested that a single dimension explained the data reasonably well. There were no indications of floor or ceiling effects. Cronbach's alpha was 0.91. Correlations between the COVID-19 Fears Questionnaire and measures of anxiety (r = 0.53), depressive symptoms (r = 0.44), and perceived stress (r = 0.50) supported construct validity. CFA supported the single-factor structure (χ2(35) = 311.2, p < 0.001, Tucker-Lewis Index = 0.97, Comparative Fit Index = 0.96, Root Mean Square Error of Approximation = 0.12). Conclusion The COVID-19 Fears Questionnaire for Chronic Medical Conditions can be used to assess fear among people at risk due to pre-existing medical conditions during the COVID-19 pandemic.

The COVID-19 outbreak has transformed the lives of people around the world through its rapid spread, number of deaths, social disruption, and devastating economic impact. 1 Fear of oneself or close relatives becoming infected is common among people exposed to any infectious disease outbreak. 2 During COVID-19, there may also be widespread fear that health care systems will not have adequate capacity and that appropriate medical care will not be available if one becomes infected, that isolation will be long-lasting with a heavy toll on mental health and social functioning, and that individual and public economic resources will not be sufficient or will not recover post-pandemic. [3] [4] [5] People with chronic diseases, particularly respiratory diseases, are at risk of severe complications from COVID-19 and may be more likely to experience negative mental health outcomes. 2 A Fear of COVID-19 Scale was developed for measuring fear of COVID-19 in the general population 5 and was translated into several languages and national contexts. [6] [7] [8] [9] [10] [11] No scales, however, have been developed and validated to assess the specific fears of vulnerable individuals due to pre-existing medical illnesses.

We solicited a list of fears during the COVID-19 outbreak from people living with the rare, chronic, autoimmune disease systemic sclerosis (SSc; scleroderma). People with SSc are representative of other groups of patients who are vulnerable due to a pre-existing medical condition; they are at risk of severe complications if infected due to lung involvement, 12, 13 general frailty, 12 and the use of immunosuppressant drugs. 14 We used suggestions from 121 people with SSc and content analysis to develop a preliminary 15-item version of a fear measure for people with chronic medical conditions. 15 J o u r n a l P r e -p r o o f

The objectives of the present study were to (1) evaluate items for inclusion in the final COVID-19 Fears Questionnaire for Chronic Medical Conditions; (2) evaluate the factor structure, internal consistency reliability, and convergent validity of the questionnaire; and (3) verify the factor structure and other validity indictors in follow-up data.

This was a cross-sectional study that analyzed, separately, two waves of data from participants enrolled in the Scleroderma Patient-centered Intervention Network (SPIN) COVID- 19 Cohort. 15 , 16 We used baseline data (Wave 1) for initial validation and item selection, and data from Wave 2 (two weeks later) for verification. The SPIN COVID-19 Cohort study was approved by the Research Ethics Committee of the CIUSSS du Centre-Ouest-de-l'Île-de-Montréal.

The SPIN COVID- 19 Cohort was open for enrolment between April 9, 2020 and April 27, 2020. 15 Participants were recruited from the ongoing SPIN Cohort 17 Twitter and by sharing with patient organization partners that it was seeking to identify fears experienced by people with scleroderma during the COVID-19 crisis. 4 The announcement was posted on March 26, 2020 and accessible for 72 hours. Respondents were directed to an online Qualtrics survey that allowed them to enter between 1 and 10 fears. Respondents were instructed to, "Please list any fears you are experiencing, including things specific to scleroderma (e.g., that an infection would make my scleroderma worse) or not specific to scleroderma (e.g., that access to regular medical care will be limited or not available)." The survey was anonymous with only information on country collected. A total of 121 people provided between 1 and 10 fears. Original survey data can be found at https://osf.io/ka43f/. 4 We employed content analysis to categorize fears into common themes to support item development. 20 One investigator (BDT) initially read all responses and generated a set of initial item themes. Then, that investigator and 12 members of the research team together reviewed fears listed by 10 respondents, classified them into initial item themes, and discussed a coding approach. Next, suggestions from the remaining 111 respondents were divided among research team members to complete classification. We involved 12 team members to review all responses carefully in a short time period so that the measure could be integrated into the SPIN COVID-19

Cohort. BDT reviewed all codes. (nearly every day) with higher scores (range 0 to 24) indicating more depressive symptoms. The PHQ-8 performs equivalently to the PHQ-9, 27 which is a valid measure of depressive symptoms in patients with SSc. 29 The PHQ-8 is available in English and French. 29 Perceived Stress: The 10-item Perceived Stress Scale (PSS) 30 measures the degree to which respondents appraise their life circumstances in the previous 4 weeks as unpredictable, uncontrollable, or overloaded. Items are scored on a 5-point scale from 0 (never) to 4 (very often). Total scores (range 0 to 40) are computed by summing individual item scores, and higher scores reflect greater perceived stress. The PSS has been validated in many medical and nonmedical populations, 31 including in France. 32 For the SPIN COVID-19 Cohort, the PSS was adapted to query about the perceived stress in the last week rather than the last 4 weeks.

Descriptive statistics were calculated as the mean and standard deviation (SD) for continuous variables and frequencies and percentages for categorical variables. Means, SDs, item intercorrelations, and corrected item-total correlations were calculated for each item of the COVID-19 Fears Questionnaire, and the mean and SD was calculated for the total score. Floor and ceiling effects were examined, defined as ≥ 15% of the participants having the lowest or highest possible score, respectively. 33, 34 Cronbach's alpha was calculated to assess internal consistency.

Exploratory factor analysis (EFA) was conducted to identify the number of factors and assess item factor loadings to inform item selection. 35 EFA was done using weighted least squares mean with variance adjusted estimation, which accounts for the ordinal nature of the J o u r n a l P r e -p r o o f survey items, and with conventional standard errors, chi-square test statistic, and geomin oblique rotation. 36 Cattell's scree test on the sedimentation graph was examined. The number of factors was chosen based on the scree plot (eigenvalues), model adequacy, and overall interpretability.

Model adequacy was assessed using a chi-square goodness-of-fit test and three fit indices, including the Tucker-Lewis Index (TLI), 37 the Comparative Fit Index (CFI), 38 

Square Error of Approximation (RMSEA). 39 Since the chi-square test is highly sensitive to sample size and can lead to the rejection of well-fitting models, practical fit indices were emphasized. 40 Models with a TLI and CFI close to 0.95 or higher, and RMSEA close to 0.06 or lower are representative of good fitting models. 41 A CFI of 0.90 or above 42 and a RMSEA of 0.08 or more 43 may also be considered to represent reasonably acceptable model fit. Items were considered for removal if (1) the corrected item-total correlation was lower than 0.5; 44, 45 (2) the factor loading was lower than 0.6; 44, 45 or (3) there were high conceptual overlap/redundancy with other items. 44 To examine convergent validity, hypotheses on the direction and magnitude of Pearson's correlations with other psychological outcome measures were formulated a priori, based on existing evidence on fear during pandemic, 2,3 generally, and for fear of progression, measured with a different scale, in SSc. 46 The magnitude of correlations was interpreted as small (|r| ≤ 0.3), moderate (0.3 < |r| < 0.5), or large (|r| ≥ 0.5). We expected to obtain moderate to large positive correlations of the COVID-19 Fears Questionnaire with anxiety symptoms, depressive symptoms, and perceived stress.

Confirmatory factor analysis (CFA) was performed to confirm the factor structure of the COVID-19 Fears Questionnaire using Wave 2 data. The CFA used the weighted least squares estimator with a diagonal weight matrix, robust standard errors, and a mean-and variance-J o u r n a l P r e -p r o o f adjusted chi-square statistic with delta parameterization in Mplus 7. 17 Model adequacy was assessed using a chi-square test, TLI, 37 CFI, 38 and RMSEA. 39 For a one-factor CFA with 8 indicators, the minimum required sample size is estimated to be between 30 and 90, assuming factor loadings between 0.50 and 0.80. 47 There were 10 indicators in the present study, and a model with more indicators requires even smaller sample relative to models with fewer indicators. 47 Stable estimates of correlations are typically achieved with a sample size of 250 or greater, although smaller correlations require larger samples. To assess a Pearson's correlation with 95% confidence and a precision of 0.10, a sample size of ≥ 403 is required for a correlation of 0.30, and ≥ 275 for a correlation of 0.50. 46 Based on sample size requirements for CFA and correlation analyses, the available number of patients 563 from the Wave 2 dataset was more than sufficient.

EFA and CFA were conducted using Mplus 7, and all other statistical analyses were conducted using SPSS (Version 25).

In total, 800 participants were included in the SPIN COVID-19 Cohort. Of these, 13 did (Table 1) .

Mean and SD of item scores are shown in Table 2 . Mean item scores ranged from 1. Table 2 ).

J o u r n a l P r e -p r o o f EFA of the 15-item preliminary COVID-19 Fears Questionnaire yielded two eigenvalues greater than one (Factor 1 Eigenvalue 8.9 and Factor 2 Eigenvalue 1.2). Based on examination of the scree plot and item factor loadings, we judged that a one-factor solution provided the most interpretable model, as the two-factor model (inter-factor correlation = 0.66) had many items with substantial cross-loadings and was not readily interpretable. Model fit for the one-factor solution was good based on the CFI and TLI, although suboptimal based on the RMSEA ( 2 (90) =1227.1, p < 0.001; CFI = 0.96; TLI = 0.95; RMSEA = 0.13). Based on inspection of the factor loadings, item correlations, and potential item redundancy due to conceptual overlap, items 3, 5, 6, 8 and 15 were removed, as per our predetermined criteria ( Table 2) .

Questionnaires for Chronic Medical Conditions (see Figure 1 ). The mean (SD) of the 10-item COVID-19 Fears Questionnaire total score was 28.0 (9.7) (median = 28.0, range 10.0 to 50.0, skewness = 0.16, kurtosis = -0.85). As shown in Table 2 Table 3 .

As shown in Table 4 , there were moderate to large correlations between the COVID-19

Fears Questionnaire and measures of anxiety (r = 0.53), depressive symptoms (r = 0.44), and perceived stress (r = 0.50). All correlations were consistent with convergent validity hypotheses.

All hypotheses were also confirmed in the 11-item COVID-19 Fears Questionnaire for Systemic Sclerosis (Appendix C), including the additional item on immunosuppressant drugs use.

CFA was performed on the remaining 10 items to confirm the single-factor structure of the fear questionnaire using Wave 2 data. In the initial CFA, in which measurement errors between all items were specified as uncorrelated, model fit for the hypothesized single-factor model was suboptimal (χ 2 (35) = 541.6, p < 0.001, TLI = 0.94, CFI = 0.95, RMSEA = 0.16).

Inspection of the modification indices indicated that model fit would be improved if the error terms of Items 1 and 14, Items 9 and 10, and Items 12 and 13 were freed to covary. Items 1 ("I will become infected when I have to leave the house to get supplies or when supplies are brought to me") and 14 ("I will be infected with the virus") both measure fear of being infected with COVID-19. Items 9 ("I will be infected and will not receive the medical treatment I need") and 10 ("I will be infected and healthcare professionals will not be familiar with the needs of a person with my condition") both evaluate the fear of medical treatment not meeting (diseasespecific) needs. Items 12 ("I will not be able to access medications I need for my scleroderma due to shortages") and 13 ("I will not be able to obtain basic supplies (e.g., food, other household necessities)" both assess fear of shortages in supplies. Therefore, the model was refitted to the J o u r n a l P r e -p r o o f data, allowing the error terms of these items to covary. These changes resulted in improvements in model fit (χ 2 (32) = 311.2, p < 0.001, TLI = 0.96, CFI = 0.97, RMSEA = 0.12). All factor loadings were adequate, with factor loadings ranging from 0.68 (item 13) to 0.89 (item 7).

Results of the 10-item CFA are shown in Table 2. CFA was also performed to confirm the single-factor structure of the 11-item COVID-19

Fear Questionnaire for Systemic Sclerosis (model including the error terms freed to covary as in the general version). Model fit for the single-factor structure was comparable to the 10-item 

The COVID-19 Fears Questionnaire for Chronic Medical Conditions is the first measure assessing pandemic-related fears among patients vulnerable due to pre-existing illnesses. 4 The main findings of this study were that the 10-item measure can be scored with a total score reflecting a single dimension and that the scale had good internal consistency reliability and convergent validity.

In addition to the COVID-19 Fears Questionnaire for Chronic Medical Conditions, we tested a SSc-specific version, which included an additional item that reflected fears of having to discontinue the use of immunosuppressant medications, which are used by approximately half of people with SSc. 48 The measurement properties did not change meaningfully by inclusion or exclusion of the item, but some patient advisors and team members believed that content validity of a measure for people with SSc required coverage of this topic. It may be the case that investigators who conduct studies of people with other medical conditions with disease-specific J o u r n a l P r e -p r o o f fears may also explore whether there are disease-specific aspects that may be added; however, in the present study, we do not believe that this is a requirement for valid measurement of fears in COVID-19.

A recently published paper reported on the development and initial validation of a general measure for fear in COVID- 19. 5 For that measure, items were derived from multiple existing fear measures and selected by experts for inclusion. There was no input, however, from members of the public. Items from general fear measures were adapted by adding that manifestations of fear during COVID-19 and included items on being afraid; discomfort thinking about the pandemic; clammy hands; being afraid of losing life; nervousness and anxiety when watching news; inability to sleep; heart racing or palpitating. Except for one item on fear that life could be lost, items reflect cognitive and physiological manifestations of general anxiety and fear not specific to COVID-19. The COVID-19 Fears Questionnaire for Chronic Medical Conditions, on the other hand, was designed to evaluate level of fear about specific aspects of the pandemic, such as fear for long-term social isolation, shortage of basic supplies, potential medical complications, and inability to access health care or medication needed for pre-existing conditions. As such, it is modelled generally on a disease-or vulnerability-specific approach; we referred to a measure on fear of progression for the general structure, 46 and we developed items based on patient input.

Because of these core differences in the approach and focus of the measures, we believe that the COVID-19 Fears Questionnaire for Chronic Medical Conditions may be a more appropriate and specific measure for evaluating fear due to COVID-19 and its consequences for people with preexisting medical diseases, although the two measures should be compared in future studies.

The COVID-19 Fears Questionnaire for Chronic Medical Conditions can be used to better evaluate behaviours and other outcomes among people with medical conditions during the J o u r n a l P r e -p r o o f current pandemic situation and as the situation develops over time. For example, fear due to COVID-19 is one of the outcome measures in trial of a mental health intervention designed to reduce anxiety among people with SSc. 15 Having a valid fear measure specifically designed for people with chronic diseases will allow exploration of how COVID-19-related fear is associated with other mental health outcomes, such as anxiety, which will provide information on how best to tailor future interventions for people living with pre-existing medical conditions during pandemic situations.

An important strength of the present study was that items were generated based on fears shared by over 100 people with SS, even though the measure was developed quickly and there Questionnaire. The demographic (including gender, ethnicity, and country) and medical characteristics of SPIN-COVID-19 Cohort participants are similar, though, to our ongoing SPIN Cohort, which is comparable with other large international SSc cohorts. 13 Second, this study only included patients with SSc, and item development did not include participants with other chronic medical conditions. Ideally, the scale will undergo further evaluation in other patient groups. In addition, although the study was conducted in a longitudinal cohort, we confined our analyses to cross-sectional analyses. We did not evaluate consistency over time because the natural history and degree of stability of fear during COVID-19 were unknown. Finally, in different settings, it J o u r n a l P r e -p r o o f is possible that some questionnaire items may be perceived differently than in SSc or that parts of items may overlap. Future studies could further test the applicability of items and consider reformulation based on setting and population.

To conclude, results of the present study demonstrate that the 10-item COVID-19 Fears 

Below is a list of statements that are related to COVID-19 and possible fears you may have about its consequences. Please select the response that reflects how much each statement describes your experience on a typical day in the last week. You may choose among not at all, slightly, moderately, very, extremely. Please do not skip any questions.

Not at All Slightly Moderately Very Extremely 1 I will become infected when I have to leave the house to get supplies or when supplies are brought to me 2 I will not be able to access health care that I need for my condition 3

I will need to be isolated for longer than others because of my condition 4

I will be infected and experience more severe complications because of my condition 5

I will be infected and will not receive the medical treatment I need 6

I will be infected and healthcare professionals will not be familiar with the needs of a person with my condition 7

People close to me (e.g., family, close friends) will be infected and become ill 8 I will not be able to access medications I need for my condition due to shortages 9 I will not be able to obtain basic supplies (e.g., food, other household necessities) 10 I will be infected with the virus The Scleroderma Patient-centered Intervention Network.

J o u r n a l P r e -p r o o f Table 1 J o u r n a l P r e -p r o o f 

The coronavirus pandemic in five powerful charts

The psychological impact of quarantine and how to reduce it: rapid review of the evidence

Fear in the time of covid

Preliminary COVID-19 Fears Questionnaire: Systemic Sclerosis and Chronic Medical Conditions Versions

The Fear of COVID-19 Scale: Development and Initial Validation

Psychometric Validation of the Bangla Fear of COVID-19 Scale: Confirmatory Factor Analysis and Rasch Analysis

Fear of COVID-19 scale: Psychometric characteristics, reliability and validity in the Israeli population

COVID-19 Fear in Eastern Europe: Validation of the Fear of COVID-19 Scale

Validation and Psychometric Evaluation of the Italian Version of the Fear of COVID-19 Scale

Adaptation and evaluation of Turkish version of the fear of COVID-19 Scale

Adaptation of the Fear of COVID-19 Scale: Its Association with Psychological Distress and Life Satisfaction in Turkey

Systemic sclerosis

The Scleroderma Patient-Centered Intervention Network Cohort: baseline clinical features and comparison with other large scleroderma cohorts

Treatment outcome in early diffuse cutaneous systemic sclerosis: the European Scleroderma Observational Study (ESOS)

Protocol for a partially nested randomised controlled trial to evaluate the effectiveness of the Scleroderma Patient-Centered Intervention Network COVID-19 Home-Isolation Activities Together

Program to reduce anxiety among at-risk scleroderma patients

Protocol for the Scleroderma Patient-centered Intervention Network (SPIN) COVID-19 Cohort and Analysis of Changes in Symptoms of

Anxiety and Depression from Pre-COVID-19 to COVID-19. OSF Preprints. 2020

The Scleroderma Patient-centered Intervention Network (SPIN) Cohort: protocol for a cohort multiple randomised controlled trial (cmRCT) design to support trials of psychosocial and rehabilitation interventions in a rare disease context

Classification criteria for systemic sclerosis: an American College of Rheumatology/European League Against Rheumatism collaborative initiative

Qualitative content analysis

World Health Organization. Process of translation and adaptation of instruments

PROMIS measure development and research

The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks

Validity of two new patient-reported outcome measures in systemic sclerosis: Patient-Reported Outcomes Measurement Information System 29-item health profile and Functional Assessment of Chronic Illness Therapy-Dyspnea Short Form

Validation of the Patient-Reported Outcomes Measurement Information System-29 (PROMIS-29) in scleroderma and associations with clinical characteristics: a Scleroderma Patient-centered Intervention Network (SPIN) Cohort study

The PHQ-8 as a measure of current depression in the general population

Equivalency of the diagnostic accuracy of the PHQ-8 and PHQ-9: a systematic review and individual participant data meta-analysis

Comparison of the PHQ-9 and CES-D depression scales in systemic sclerosis: internal consistency reliability, convergent validity and clinical correlates

Are scores on English and French versions of the PHQ-9 comparable? An assessment of differential item functioning

A global measure of perceived stress

Review of the psychometric evidence of the Perceived Stress Scale

Psychometric properties of the French versions of the Perceived Stress Scale

Quality criteria were proposed for measurement properties of health status questionnaires

COSMIN standards and criteria for evaluating the content validity of health-related Patient-Reported Outcome Measures: a Delphi study

Mplus user's guide version

Evaluating the use of exploratory factor analysis in psychological research

A reliability coefficient for maximum likelihood factor analysis

Comparative fit indexes in structural models

Structural model evaluation and modification: an interval estimation approach

Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance

Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives

Principles and practice of structural equation modeling

Alternative ways of assessing model fit

Psychometric Theory: 2nd Ed. McGraw-Hill

Multivariate data analysis: A global perspective

Validity of the Fear of Progression Questionnaire-Short Form in patients with systemic sclerosis

Sample size requirements for structural equation models: an evaluation of power, bias, and solution propriety

Comparison of mental health symptoms prior to and during COVID-19 among patients with systemic sclerosis from four countries: a Scleroderma Patient-centered Intervention Network (SPIN) Cohort study

We thank the 121 people with scleroderma who provided item suggestions for the questionnaire.

The study was supported with funding from the Canadian Institutes of Health Research

 Figure 1