key: cord-0741338-dgpkv7b6 authors: So, Christine J.; Kim, Jinu; Cifre, Anthony B.; Gonzalez, Rogelio D.; Bower, Joanne L.; Alfano, Candice A. title: Validation of the Mental Health Checklist (MHCL) during COVID‐19 lockdown date: 2021-05-17 journal: J Community Psychol DOI: 10.1002/jcop.22600 sha: 86dc3e5f21a5fefbc55f67b2f04a6e9e35dd8d5d doc_id: 741338 cord_uid: dgpkv7b6 The COVID‐19 pandemic resulted in unprecedented disruption to everyday life, including widespread social distancing and self‐quarantining aimed at reducing the virus spread. The Mental Health Checklist (MHCL) is a measure developed to assess psychological health during extended periods of isolation and confinement, and has shown strong psychometric properties in community samples and during Antarctic missions. This study validated the MHCL in a sample of 359 U.S. and U.K adults during the peak of the COVID‐19 lockdown. Confirmatory factor analysis (CFA) tested model fit, and convergent validity analyses were conducted to compare the MHCL with validated measures of depression, anxiety and stress, as well as insomnia. The MHCL exhibited good model fit for most CFA indices, and showed strong convergent validity with other measures of psychological well‐being. Findings suggest that the MHCL is useful for assessing mental health in a variety of environments and conditions. confinement worldwide Tran et al., 2020) . Despite a fundamental goal of protecting public health, such confinement and isolation (e.g., from family and friends) fall outside of the typical human experience, imposing risk for various types of problems including anxiety and panic, depression and irritability, maladaptive health behaviors, and sleep disruption (see Zvolensky et al., 2020) . At the same time, the manifestation of mental health problems in the face of elevated stress varies considerably across individuals and there is little understanding at present of which persons are most susceptible. As critical first step, validated tools for monitoring and evaluating psychological health during periods of isolation and confinement are essential. The COVID-19 pandemic, which first spread to the United States and United Kingdom in early 2020, has significantly altered life for all citizens. Toward slowing the spread of this disease, unprecedented public health measures have been taken including shelter-in-place orders, social distancing, closing of businesses, shifting schooling and work online, canceling of local, state and national events, and bans on long-distance travel. Although such extreme measures offer the best defense against the virus in the absence of an effective vaccine, they are not without negative effects of their own. Emerging studies almost unanimously document the deleterious impact of the COVID-19 pandemic on mental and physical health outcomes, including increased rates of probable anxiety and depressive disorders (Gallagher et al., 2020; Rossi et al., 2020) , trauma/ stressor-related symptoms (Czeisler et al., 2021; Tan et al., 2020) , insomnia (Rossi et al., 2020) , substance use (Czeisler et al., 2021) and suicidal ideation (Czeisler et al., 2021) . Less clear, however, is the contribution of specific aspects of the pandemic to mental health problems. For example, some research has identified linear relationships between the amount of time a person spends at home self-quarantining and psychological risk (Brooks et al., 2020) , while other data have not found a relationship between social isolation and mental health burden during the pandemic (Milman et al., 2020a (Milman et al., , 2020b . Another recent study found that living with six or more people in the household was a protective factor against symptoms of depression and anxiety (Wang et al., 2021) . This finding may indicate that, while the number of individuals in a household may be protective to an extent, other pandemic-related factors may be more strongly associated with adverse outcomes. For instance, mental health risk might be directly elevated by uncertainty related to and imposed by the pandemic, for example perceptions of how long the pandemic will continue, worry about friends and/or loved ones becoming sick, and job/wage loss. In the absence of validated measures for assessing psychological health during periods of isolation and confinement, our group developed the Mental Health Checklist (MHCL; Alfano et al., 2021; Bower et al., 2019) which yields three reliable subscales measuring, positive adaptation, poor self-regulation, and anxious apprehension. Strong reliability and convergent validity were found for the three MHCL subscales among crew members at two Antarctic stations, both upon arrival on station (Bower et al., 2019) and across several consecutive winter months (Alfano et al., 2021) . The Antarctic represents one of the most extreme and stressful environments on earth, characterized by confinement due to harsh weather conditions, social isolation, and no or limited opportunity for departure during the winter months. As such, it remains unknown whether the MHCL subscales provide reliable assessment of mental health risk in other environments, such as during a period of imposed quarantine and isolation within one's own home, as elicited by the COVID-19 pandemic. In the current study, we examined the factor structure and psychometric properties of the MHCL among adults in the United States and United Kingdom between April and May 2020, approximately 1 month after mandatory shelter-in-place and social distancing orders were implemented in the U.K and most U.S. states. Specifically, we conducted a confirmatory factor analysis of the MHCL and examined convergent validity with validated measures of depression anxiety, and stress. Further, given robust associations between emotional well-being and sleep patterns (Harvey et al., 2011) and reports of increased insomnia symptoms during COVID-19 (e.g., Rossi et al., 2020) we examine convergent validity of the MHCL with a validated measure of insomnia. Last, we examined relationships between the MHCL and unique aspects of the pandemic including the extent to which participants reported staying at home and socially-isolating, beliefs about how long the pandemic would continue, and worry about friends and/or loved ones becoming sick and job/wage loss. Participants were recruited for online survey about coping during the COVID-19 pandemic, and predominantly came from the United States (n = 344) with a smaller subset from the United Kingdom (n = 15). A total of 429 participants met inclusion criteria and provided consent to participate. From this sample, 359 completed the entire survey (completion rate = 83.7%) and were included in this study. Inclusion criteria included: a) U.S. or U.K. resident; (b) age 21-50 years; and (c) proficiency in English. Participants were excluded if they were currently or were previously infected with or suspected to have had the coronavirus. The mean age of the participants was 37 years (SD = 8.53), with most participants identifying as female (81.1%) and White (74.4%). The majority of participants reported having a college degree or higher (75%) and being married (47.6%). Additionally, the average number additional individual living in the household was around 2, while only 12% reported living alone. See Table 1 for demographic characteristics and Table 2 for COVID-19-related characteristics. Participants were recruited through social media platforms (e.g., Facebook, Twitter) between April 17 and May 14, 2020. This 4-week period was approximately 1 month after shelter-in-place and social distancing orders were implemented in the U.K. and most U.S. states. All measures were completed anonymously via an online platform (Qualtrics). Recruitment advertisements provided the link to the online survey as well as the inclusion and exclusion criteria. The landing page of the survey included detailed information about and objectives of the study and required participants to agree to continue. Consent was followed by several screening questions to ensure eligibility. Participants then answered questionnaires in the order they appear below. The online survey required approximately 15 min to complete, and following completion, participants were given an option to enter into a lottery to win one of four $25 gift cards. The Institutional Review Board at the University of Houston approved all study procedures. Participants' age, sex, gender, race/ethnicity, marital status, household occupancy, employment status, status of current shelter-in-place/stay-at-home order in their area, initial date they began staying at home, and individual/ health-related risk factors for COVID-19 were collected (e.g., being a healthcare or essential worker). Additionally, participants reported if they had a family member or close friend that has been diagnosed with and/or died from COVID-19. Participants completed several questions regarding their behavior and worries since the start of the pandemic, including the extent to which they were staying at home and social distancing on an 11-point Likert-type scale (0 = "not at all" to 10 = "all of the time"). Participants also rated their degree of worry about whether someone in SO ET AL. their household would contract the virus and about losing a job or wages due to the pandemic. Worries were also rated on 11-point Likert-type scale (0 = "none" to 10 = "extremely"). | 5 11-point Likert-type scale ranging from 0 ("never") to 10 ("always") and summed to generate three subscale scores: positive adaptation (e.g., in full control, inspired, determined), poor self-regulation (e.g., restless/fidgety, inattentive, sleepy), and anxious apprehension (e.g., worried, obsessional/stuck on things). The MHCL has been shown to possess acceptable psychometric properties for use in isolated, confined and stressful environments (Bower et al., 2019) . All three MHCL scales exhibited good internal consistency in the current sample (α's = .78-.90). Descriptive statistics for individual items on the MHCL are presented in Table 3 . The Depression, Anxiety, Stress Scale-21 (DASS-21; Henry & Crawford, 2005) , originally abbreviated from the DASS-42 (Lovibond & Lovibond, 1995) is a measure assessing the tripartite model of psychopathology: depression, anxiety, and stress. Items are rated on a 4-point Likert-type scale that ranges from 0 ("did not apply to me at all") to 3 ("applied to me most of the time or always"). Examples of the items included, "I felt I had nothing to look forward to," "I felt down-hearted and blue," and "I felt that life was meaningless." The DASS-21 has been validated in both clinical (Clara et al., 2001) and nonclinical samples (Sinclair et al., 2011) , and has also been used in several COVID-19 studies across different cultures (e.g., Wang et al., 2020 Wang et al., , 2021 . Excellent reliability for the full-scale was found the current sample (α = .93) as well as for each subscale (α's = .83-.90). To compare scores in the current sample to scores from the full DASS-42 scale, scores were doubled (Henry & Crawford, 2005 ). The insomnia severity index (ISI; Bastien et al., 2001 ) is a 7-item measure that evaluates self-reported severity of insomnia symptoms-sleep maintenance, early awakening, sleep-onset, functional impairment, sleep pattern satisfaction, noticeability of sleep-related impairments, and general distress caused by sleep problems. The ISI utilizes a 5-point Likert-type scale ranging from 0 to 4, with higher scores indicating more severe symptoms of insomnia. The ISI has been used in previous COVID-19 studies (e.g., Killgore et al., 2020) and exhibited excellent reliability in the current sample (α = .90). Preliminary analyses were conducted using IBM SPSS version 26. Data were first examined to ensure that only genuine responses were included. Cases were removed if participants omitted any items from the MHCL, DASS, or ISI. Sample descriptive statistics were computed for each MHCL item, as well as MHCL, DASS, and ISI subscales and total scores. CFA was conducted using IBM AMOS version 26. Items from the MCHL were first assessed for normality, and skewness and kurtosis values were within acceptable ranges. Model fit for CFA was assessed using the following indices: (a) χ 2 , though this is mainly reported for information purposes since the test is overpowered in moderate-large sample sizes; (b) comparative fit index (CFI), a measure of incremental fit most suitable for small sample sizes; root mean square error of approximation (RMSEA), a noncentrality measure of fit that is a function of chi-square fit; and standardized root mean square residual (SRMR) a measure of absolute fit less sensitive to sample size than RMSEA. Good model fit is indicated by an insignificant χ 2 test (p > .05), CFI ≥ .90, SRMR < .08, and RMSEA < .08 (Chen, 2007; Hooper et al., 2008; Hu & Bentler, 1999; Kline, 2005) . Finally, Pearson's correlations were examined to assess the degree to which each MHCL subscale corresponded with specific pandemic-related behaviors and worries. The overwhelming majority of the sample (94.7%) reported currently living under a "Stay/Safer/Shelter at Home" order in which nonessential businesses and schools were closed due to the pandemic. Most participants reported staying at home starting in mid-March 2020, approximately 1 month before the start of data collection. Almost 1/3rd of participants (32.9%) reported losing a job or wages as a result of the pandemic, and 18.4% reported having a family member or close friend who was diagnosed with COVID-19. When queried about how worried participants were that they or someone in their household would contract COVID-19 from 0 ("none") to 10 ("extremely"), the average rating was 5.95 (SD = 2.74). When queried about worrying whether they would lose a job or wages due to COVID-19, participants reported an average rating of 5.28 (SD = 3.66). Descriptive statistics for MHCL subscales, DASS and ISI for the entire sample are presented in Table 4 . Descriptive statistics for individual MHCL items are presented in Table 3 . The χ 2 goodness of fit test was statistically significant (χ 2 = 809.01, p < .001; relative χ 2 = 3.56). The CFI value of the model was 0.85, just below the criteria for good fit of more than 0.90. Measures of absolute fit suggested the model to be acceptable. The RMSEA value was 0.08 (90% confidence interval = 0.08-0.09) and the SRMR value was 0.07 (see Figure 1 ). Correlations between DASS subscales, total ISI score, and MHCL subscales are presented in Table 3 Correlations between the MHCL subscales and unique aspects of the pandemic are presented in Table 5 . The positive adaptation subscale of the MHCL was negatively correlated with the extent to which participants were staying at home (r = −.18, p = .001), worries about someone in the household contracting COVID-19 (r = −.15, p = .004), and worries about losing a job or wages due to COVID-19 (r = −.14, p = .01) though correlations were relatively small. The MHCL poor self-regulation subscale was not significantly correlated with any pandemic-related behaviors (i.e., staying at home or social distancing), but was positively correlated with worries about contracting the disease (r = .14, p = .01) and worries about losing a job/wages (r = .18, p = .001). Likewise, the anxious apprehension subscale was not significantly correlated with pandemic-related behaviors, but was positively correlated with worries about contracting COVID-19 (r = .29, p < .001) and worries about losing a job/wages (r = .30, p < .001). Notably, the number of people living in the home, the total number of at-risk groups, and perceptions of how long the pandemic would last were not significantly associated with any MHCL subscale. The somewhat mixed in that not all indices suggested good fit for our model. For instance, the χ 2 value was statistically significant, suggesting a discrepancy between the sample and covariance matrices. However, this finding is not surprising since, as previously noted, this test is overpowered in moderate-to-large sample sizes. Comparatively, the observed relative chi-square value is suggestive of adequate model fit (i.e., below 5; Schumacker & Lomax, 2004) . The CFI value fell just below the acceptable fit criterion, whereas estimates of absolute fit (i.e., RMSEA and SRMR) indicated better fit. These discrepancies are likely due to differential functions of these indices; for instance, given the appropriateness of CFI and RMSEA for small and large sample sizes, respectively, CFI has been suggested as better suited for exploratory factor analyses, while RMSEA is better suited for confirmatory factor analyses (Rigdon, 1996) . Based on these guidelines, model fit in the present study would be viewed as acceptable. Overall, our data indicate MHCL as a potentially useful measure of mental health during periods of isolation and confinement at home. The MHCL showed strong convergent validity with measures of depression, anxiety, stress, and insomnia symptoms. Specifically, MHCL poor self-regulation and anxious apprehension subscales were positively associated with DASS and ISI scores, whereas MHCL positive adaptation scores were negatively related with these scores, albeit based on smaller correlations. The latter finding is not surprisingly in light of the fact that the DASS and ISI measure negative experiences/symptoms and positive emotions are not simply the inverse of negative ones. Rather, whereas negative emotions tend to elicit a narrow range of cognitions and behaviors, positive emotions are known to broaden thought-action repertoires, prompting a wider array of thoughts and actions (Fredrickson, 1998) . Positive emotions have nonetheless received limited attention in the context of isolation and confinement, including whether they might serve to buffer against the negative effects of stress. In this sense, the MHCL might offer advantage for assessing mental health risk in high-stress settings/circumstances. We also examined the extent to which the MHCL subscales correlated with specific COVID-related behaviors and worries. Only the positive adaptation subscale was significantly (negatively) associated with extent to which participants were staying at home, potentially suggesting positive emotions to be more susceptible to the psychological effects of quarantining at home, at least in the short term. Conversely, we observed positive associations between the negative symptom MHCL subscales and worries about the pandemic, similar to results reported by Taylor et al. (2020) where high fear beliefs (e.g., "dangers of COVID-19") were positively associated with general distress and avoidance. These authors posited that those with higher anxiety may be more likely to engage in excessive avoidance behaviors . We did not measure behavioral avoidance specifically in the current study, but we did find COVID-related worries to correlate with greater self-isolating and social distancing. It is important to note that the MHCL was originally developed for use in extreme, stressful environments (e.g., polar environments, space flight). In the Antarctic, for example, individuals face extended periods of indoor confinement, altered light-dark cycles, monotony, forced closeness with others, separation from family and friends, and limited ability or a complete inability to depart from the continent; conditions that increase risk for a range of psychological problems, interpersonal stress, and sleep disturbance (Palinkas & Suedfeld, 2008) . While conditions created by the COVID-19 pandemic are not comparable to the Antarctic, the virus has resulted in unprecedented circumstances and extreme stress for many. Beyond, mandatory social isolation and confinement at home, ominous messages from media, hospitals at capacity, exponential rates of infection and mortality, unemployment rates not seen in the United States since the Great Depression (Bureau of Labor Statistics, 2020), the closing of schools and businesses, and an overall uncertain duration render the pandemic extreme in its own right, and necessitate research focused on understanding and identifying associated mental health risk. Several limitations warrant consideration in our study. Although our sample was larger and more diverse compared to our previous validation studies (Bower et al., 2019) , the participants of this study were predominately educated, Caucasian females. This is in direct contrast to our Antarctic validation sample (Alfano et al., 2021 , Bower et al., 2019 who were predominantly male. This fact alone might partially explain mixed evidence for a good model fit. Gender-based differences in prevalence rates of mental health symptoms, such as depression (Lim et al., 2018) and anxiety (McLean et al., 2011) , have been well-documented, including differences in how men and women present and cope with their symptoms. This may lead to gender-specific coping strategies during unexpected periods of high stress. It is also possible that those who opted to take our survey were experiencing particularly high levels of pandemic-related stress. Additionally, although data were collected during roughly the second month of shelter-in-place orders, a point at which all forms of stress were on the rise, states/regions in the United States and United Kingdom differed with regard to specific public health measures issued. Our one-time measure also did not capture day-to-day changes in worries, symptoms and stress, which may be considerable. In summary, while much remains to be understood about the effects of a prolonged, global pandemic on the mental health of citizens, such understanding hinges on the use of reliable self-report measures. Results from our preliminary study suggest the MHCL subscales may be a useful measure for this purpose. In addition to support for the measure's factor structure, we found strong convergent validity with previously-validated measures of depression, anxiety, stress and insomnia. Of the three MHCL subscales, only positive adaptation scores were significantly associated with the extent to which participants were staying at home, whereas all three subscales correlated significantly with worries about the pandemic. As the MHCL is freely available for use, we hope that other researchers will consider it for use in future studies assessing mental health in extreme environments or conditions. Mental health, physical symptoms and biomarkers of stress during prolonged exposure to Antarctica's extreme environment Validation of the Insomnia Severity Index as an outcome measure for insomnia research Factor structure and validation of the mental health checklist (MHCL) for use in isolated, confined and extreme environments The psychological impact of quarantine and how to reduce it: Rapid review of the evidence Sensitivity of Goodness of Fit Indexes to lack of measurement invariance Confirmatory factor analysis of the Depression-Anxiety-Stress Scales in depressed and anxious patients Early public adherence with and support for stay-at-home COVID-19 mitigation strategies despite adverse life impact: a transnational cross-sectional survey study in the United States and Australia What good are positive emotions? The impact of covid-19 experiences and associated stress on anxiety, depression, and functional impairment in american adults Sleep disturbance as transdiagnostic: Consideration of neurobiological mechanisms The short-form version of the Depression Anxiety Stress Scales (DASS-21): Construct validity and normative data in a large non-clinical sample Structural equation modelling: Guidelines for determining model fit Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives Suicidal ideation during the COVID-19 pandemic: The role of insomnia Anxiety and depression among people under the nationwide partial lockdown in Vietnam Prevalence of depression in the community from 30 countries between The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories Gender differences in anxiety disorders: Prevalence, course of illness, comorbidity and burden of illness Social isolation and the mitigation of coronavirus anxiety: The mediating role of meaning Social isolation as a means of reducing dysfunctional coronavirus anxiety and increasing psychoneuroimmunity Psychological effects of polar expeditions CFI versus RMSEA: A comparison of two fit indexes for structural equation modeling COVID-19 pandemic and lockdown measures impact on mental health among the general population in Italy. An N= 18147 web-based survey A Beginner's Guide to Structural Equation Modeling Psychometric evaluation and normative data for the Depression, Anxiety, and Stress Scales-21 (DASS-21) in a nonclinical sample of U Is returning to work during the COVID-19 pandemic stressful? A study on immediate mental health status and psychoneuroimmunity COVID stress syndrome: Concept, structure, and correlates Impact of COVID-19 on economic well-being and quality of life of the Vietnamese during the national social distancing The association between physical and mental health and face mask use during the COVID-19 pandemic: A comparison of two countries with different views and practices The impact of COVID-19 pandemic on physical and mental health of Asians: A study of seven middle-income countries in Asia Psychological, addictive, and health behavior implications of the COVID-19 pandemic All authors certify responsibility for the manuscript, and none of this article will be published or submitted elsewhere.